Skip to content

Node Backups

Kaleido nodes can be configured with a backup configuration, offering an on demand gateway for exporting node data into a customer’s cloud storage target. Backups not only provide an added persistence layer for the ledger, but also allow customers to access the otherwise unsurfaced signing keys and chain data and take full possession of their node contents. The following cloud storage types are supported as node backup targets:

  • AWS S3 Bucket
  • Azure Blob Storage

Customer Cloud Account Configuration

AWS S3 bucket

Configure an S3 Bucket in AWS Console

  • Log into the AWS console and navigate to the S3 Service in the Storage category.
  • Provide a compliant DNS name for your bucket and follow the pop up instructions for further configuration and permissions. Kaleido does not enforce restrictions on the bucket configuration (e.g. object encryption). Review your configurations and click Create bucket to provision the storage resource.

Configuring an IAM Role

NOTE: Support for IAM user-based longterm credentials (the AWS cloud configuration structure offered on previous Kaleido releases) is being deprecated. For newly created AWS configurations, you must use the IAM role approach outlined below.

  • Log into the AWS console and navigate to the IAM Service.
  • IAM roles will be used to grant Kaleido access to your KMS instance storing the master encryption key. Refer to the AWS guide for more information on IAM roles.
  • Click the Roles tab in the IAM navigation panel and proceed to Create a New Role.
  • Select Another AWS Account as trusted entity and use Kaleido's account ID 271815176711 in the Account ID field.
  • Skip Permissions, Tags, and finish creating role by specifying a Role Name.

Attaching an S3 Policy to your IAM Roles

  • Next, navigate to your IAM service and provision a custom S3 policy for the IAM role for which Kaleido's account (Account ID 271815176711) is the service principal. Refer to this AWS guide for more details on IAM roles.
  • In the lefthand navigation panel select Policies and click Create policy at the top of the screen
  • Choose S3 as the Service and enable write access in its entirety. This can be done by simply clicking the box next to Write.
  • Create a name for the policy and review it, ensuring that you have provided full write access. Click Create policy to finish. Minimum permissions required by Kaleido to upload node backups would be:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
            ],
            "Resource": [
                "arn:aws:s3:::<yourbucketname>/*"
            ]
        }
    ]
}
  • Next, click the Roles tab in the navigation panel to attach the newly created custom policy to the role created in previous step.
  • Click the Add Permissions button and select the Attach existing policies directly option.
  • Enter you custom policy name into the search bar and select it. Click Next: Review to finalize the user permissions and enable the custom policy against your selected role or user.

AWS S3 Pre-signed URL

Alternatively, you can generate a one-time Pre-signed URL for a file in a S3 bucket and provide that as the target for the upload of the backup. If you choose to use this mechanism to trigger a backup, the node need not be pre-configured with a S3 backup destination. This feature is only available via the API at this time. Below is a js code snippet that uses the aws-sdk nodejs package that you may use to generate a Pre-signed URL for your S3 bucket

'use strict';

const AWS = require('aws-sdk');
const Axios = require('axios');

const s3 = new AWS.S3({
  apiVersion: '2006-03-01',
  sslEnabled: true,
  accessKeyId: '<your S3 bucket access key>',
  secretAccessKey: '<your S3 bucket secret>',
  region: '<region>'
});

async function createURL(params) {
  var params = {
    Bucket: '<bucket name>',
    Key: '<file name for backup>'
  };
  let url = await s3.getSignedUrl('putObject', params);
  return url;
}

createURL()
  .then(async (url) => {
    console.log(`URL: ${url}`);
    return url;
  })
  .catch(err => {
    console.error(`Presigning request encountered an error ${err}`);
    process.exit(1);
  });

Azure Blob Storage

Configure an Azure Blob Storage account

  • Log into the Azure Portal and click on Storage Accounts.
  • Click on the Storage Account that you wish to configure as the backup target. If you need to create a Storage Account, please refer to the following documentation.
  • Click on Access Keys below Settings in the left hand navigation. You will need the Key when configuring the backup target.
  • Click on the Containers below Blob Service in the left hand navigation. Note down the Container Name that you will use as the backup target. Kaleido will upload a backup tar file to this container each time a backup operation is triggered on the node.

Configuration in Kaleido

Create Backup Configuration in Kaleido environment

The next step is to create a Backup configuration in your Kaleido environment. Backup configuration is per membership and can be attached to any nodes belonging to the membership. The Backup Configuration schema is outlined in the formal API Documentation.

You will need to use the administrative API to generate your backup configuration on Kaleido. The API response will return two unique strings - configuration_id & external_id_seed - that will be used to edit your IAM role's trust policy. You will need to assemble an external_id string with a syntax of ${configuration_id}-${external_id_seed}-external-id, where configuration_id is ID of the KMS configuration object and external_id_seed is a unique value generated by Kaleido. Both are returned in the API call to generate the KMS configuration object.

An example request to create an IAM role based backup configuration in your Kaleido environment:

https://console.kaleido.io/api/v1/consortia/:consortium_id/environments/:environment_id/configurations
{
    "membership_id": "<membership id belonging to your organization>",
    "name": "role-based-backup-config",
    "type": "backup",
    "details": {
        "provider": "aws",
        "region": "us-east-2",
        "role_arn": "<arn for the role you created to give kaleido access>",
        "bucket": "<your bucket name>"
    }   
}

Sample response:

{
    "membership_id": "<membership id belonging to your organization>",
    "name": "role-based-backup-config",
    "type": "backup",
    "details": {
        "provider": "aws",
        "region": "us-east-2",
        "role_arn": "<arn for the role you created to give kaleido access>",
        "bucket": "<your bucket name>",
        "external_id_seed": "<unique id generated by kaleido>"
    },
    "_id": "<configuration id>",
    "_revision": "0",
    "created_at": "2021-06-08T02:32:46.152Z",
    "environment_id": "<environment id>"
}

If "_id" in the above JSON response is u0jj4apxhw and "external_id_seed" is u0f5drkxrb, then the fully assembled external_id string would be u0jj4apxhw-u0f5drkxrb-external-id.

Editing your IAM Role's Trust Policy

You now need to edit the trust policy of the IAM role used in your backup configuration to include the external_id string. You can do this using following steps

  • Log into the AWS console and navigate to your IAM service.
  • Next, click the Roles tab in the navigation panel.
  • Click the Trust Relationships button and then Edit trust relationship button.
  • Update the policy document to use the external_id string as the Condition value. The updated policy document resembles the following:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::271815176711:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<external id string here>"
        }
      }
    }
  ]
}

Again, please refer to the following AWS guide to understand in detail how to use IAM roles and why they are required.

Attach Backup Configuration to Node

To attach a Backup Configuration to a Node using API, refer to the following API Documentation. You can obtain the backup_id for the relevant backup store by querying the configurations API endpoint under the environment.

To attach a Backup Configuation to a Node using the Kaleido Console, navigate to the Backup Store configuration you created in the previous step. Click on the Backup Store that you would like to associate with a node. Click on Attach Runtime at the bottom of the screen and select the Node.

Attach Backup Store

Backup Node Contents

After a backup configuration is created and associated with a node, On-demand backup request can be triggered on the node via Kaleido API. Kaleido uploads a tar zipped file of the node's contents to the AWS S3 bucket when such a backup operation is triggered on the node. Please refer to the API documentation for details on the API request to trigger a backup operation.

If you created a AWS S3 Pre-Signed URL as the backup destination, backup the node content by providing the following detail in the API request. Backup using AWS S3 Pre-Signed URL is not yet available via Kaleido Console. Use the generated Pre-Signed URL as presigned_url request body field in the API request:

{
    "presigned_url": "https://test-bucket-1.s3.us-east-2.amazonaws.com/zzver79ou5-zzvt0b0ylf-quorum.tgz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AZIAJNQZWSKAX2VQHMMA%2F20191003%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20191003T161903Z&X-Amz-Expires=900&X-Amz-Signature=cf0f8ecfc701c921b6d324007006d43977b6a67ef0911bdefa8529a78a3683ce&X-Amz-SignedHeaders=host"
}