Kaleido Open Menu
Close Menu
Log In Try It Free

Node and Service Backups

Kaleido nodes, and the Fabric CA service, can be configured with a backup configuration, offering an on demand gateway for exporting node or service data into a customer’s cloud storage target. Backups not only provide an added persistence layer for the ledger, but also allow customers to access the otherwise unsurfaced signing keys and chain data and take full possession of their node and service contents. The following cloud storage types are supported as node and service backup targets:

  • AWS S3 Bucket
  • Azure Blob Storage

Customer Cloud Account Configuration

AWS S3 bucket

Configure an S3 Bucket in AWS Console

  • Log into the AWS console and navigate to the S3 Service in the Storage category.
  • Provide a compliant DNS name for your bucket and follow the pop up instructions for further configuration and permissions. Kaleido does not enforce restrictions on the bucket configuration (e.g. object encryption). Review your configurations and click Create bucket to provision the storage resource.

Configuring an IAM Role

NOTE: Support for IAM user-based longterm credentials (the AWS cloud configuration structure offered on previous Kaleido releases) is being deprecated. For newly created AWS configurations, you must use the IAM role approach outlined below.

  • Log into the AWS console and navigate to the IAM Service.
  • IAM roles will be used to grant Kaleido access to your KMS instance storing the master encryption key. Refer to the AWS guide for more information on IAM roles.
  • Click the Roles tab in the IAM navigation panel and proceed to Create a New Role.
  • Select Another AWS Account as trusted entity and use Kaleido's account ID 271815176711 in the Account ID field.
  • Skip Permissions, Tags, and finish creating role by specifying a Role Name.

Attaching an S3 Policy to your IAM Roles

  • Next, navigate to your IAM service and provision a custom S3 policy for the IAM role for which Kaleido's account (Account ID 271815176711) is the service principal. Refer to this AWS guide for more details on IAM roles.
  • In the lefthand navigation panel select Policies and click Create policy at the top of the screen
  • Choose S3 as the Service and enable write access in its entirety. This can be done by simply clicking the box next to Write.

  • Create a name for the policy and review it, ensuring that you have provided full write access. Click Create policy to finish. Minimum permissions required by Kaleido to upload node backups would be: 

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<yourbucketname>/*"
            ]
        }
    ]
}

  • Next, click the Roles tab in the navigation panel to attach the newly created custom policy to the role created in previous step.

  • Click the Add Permissions button and select the Attach existing policies directly option.
  • Enter you custom policy name into the search bar and select it. Click Next: Review to finalize the user permissions and enable the custom policy against your selected role or user.

AWS S3 Pre-signed URL

Alternatively, you can generate a one-time Pre-signed URL for a file in a S3 bucket and provide that as the target for the upload of the backup. If you choose to use this mechanism to trigger a backup, the node need not be pre-configured with a S3 backup destination. This feature is only available via the API at this time. Below is a js code snippet that uses the aws-sdk nodejs package that you may use to generate a Pre-signed URL for your S3 bucket

'use strict';

const AWS = require('aws-sdk');
const Axios = require('axios');

const s3 = new AWS.S3({
  apiVersion: '2006-03-01',
  sslEnabled: true,
  accessKeyId: '<your S3 bucket access key>',
  secretAccessKey: '<your S3 bucket secret>',
  region: '<region>'
});

async function createURL(params) {
  var params = {
    Bucket: '<bucket name>',
    Key: '<file name for backup>'
  };
  let url = await s3.getSignedUrl('putObject', params);
  return url;
}

createURL()
  .then(async (url) => {
    console.log(`URL: ${url}`);
    return url;
  })
  .catch(err => {
    console.error(`Presigning request encountered an error ${err}`);
    process.exit(1);
  });

Azure Blob Storage

Configure an Azure Blob Storage account

  • Log into the Azure Portal and click on Storage Accounts.
  • Click on the Storage Account that you wish to configure as the backup target. If you need to create a Storage Account, please refer to the following documentation.
  • Click on Access Keys below Settings in the left hand navigation. You will need the Key when configuring the backup target.
  • Click on the Containers below Blob Service in the left hand navigation. Note down the Container Name that you will use as the backup target. Kaleido will upload a backup tar file to this container each time a backup operation is triggered on the node.

Configuration in Kaleido

Create Backup Configuration in Kaleido environment

The next step is to create a Backup configuration in your Kaleido environment. Backup configuration is per membership and can be attached to any nodes belonging to the membership. The Backup Configuration schema is outlined in the formal API Documentation.

You will need to use the administrative API to generate your backup configuration on Kaleido. The API response will return two unique strings - configuration_id & external_id_seed - that will be used to edit your IAM role's trust policy. You will need to assemble an external_id string with a syntax of ${configuration_id}-${external_id_seed}-external-id, where configuration_id is ID of the KMS configuration object and external_id_seed is a unique value generated by Kaleido. Both are returned in the API call to generate the KMS configuration object.

An example request to create an IAM role based backup configuration in your Kaleido environment:

https://console.kaleido.io/api/v1/consortia/:consortium_id/environments/:environment_id/configurations
{
    "membership_id": "<membership id belonging to your organization>",
    "name": "role-based-backup-config",
    "type": "backup",
    "details": {
        "provider": "aws",
        "region": "us-east-2",
        "role_arn": "<arn for the role you created to give kaleido access>",
        "bucket": "<your bucket name>"
    }   
}

Sample response: 

{
    "membership_id": "<membership id belonging to your organization>",
    "name": "role-based-backup-config",
    "type": "backup",
    "details": {
        "provider": "aws",
        "region": "us-east-2",
        "role_arn": "<arn for the role you created to give kaleido access>",
        "bucket": "<your bucket name>",
        "external_id_seed": "<unique id generated by kaleido>"
    },
    "_id": "<configuration id>",
    "_revision": "0",
    "created_at": "2021-06-08T02:32:46.152Z",
    "environment_id": "<environment id>"
}

If "_id" in the above JSON response is u0jj4apxhw and "external_id_seed" is u0f5drkxrb, then the fully assembled external_id string would be u0jj4apxhw-u0f5drkxrb-external-id.

Editing your IAM Role's Trust Policy

You now need to edit the trust policy of the IAM role used in your backup configuration to include the external_id string. You can do this using following steps

  • Log into the AWS console and navigate to your IAM service.
  • Next, click the Roles tab in the navigation panel.
  • Click the Trust Relationships button and then Edit trust relationship button.
  • Update the policy document to use the external_id string as the Condition value. The updated policy document resembles the following:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::271815176711:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<external id string here>"
        }
      }
    }
  ]
}

Again, please refer to the following AWS guide to understand in detail how to use IAM roles and why they are required.

Attach Backup Configuration to Node or Service

To attach a Backup Configuration to a Node using API, refer to the following API Documentation. You can obtain the backup_id for the relevant backup store by querying the configurations API endpoint under the environment.

To attach a Backup Configuration to a Service using API, refer to the following API Documentation. You can obtain the backup_id for the relevant backup store by querying the configurations API endpoint under the environment.

To attach a Backup Configuation to a Node or Service using the Kaleido Console, navigate to the Backup Store configuration you created in the previous step. Click on the Backup Store that you would like to associate with a node or service. Click on Attach Runtime at the bottom of the screen and select the Node or the Service.

Attach Backup Store

Node Status Checks

Before performing an on-demand backup of a node, it is important to ensure that the node is healthy and is in sync with the rest of the blockchain. The following steps are applicable to Ethereum based environments:

  • Check that all the nodes in the environment are healthy and in sync. For this, you may use either API or UI:

    • Environment status API call. Ensure that all nodes are reported healthy and that the maximum & minimum block heights are approximately the same (depending on the number of nodes in the environment, the time difference between polling nodes could result in small differences).
    • In the UI, click on Health & Monitoring -> Blockchain and scroll down to the Block Height panel. Ensure that there are no large differences in the block heights reported by all nodes in the environment.

  • Check that there are no Pending Transactions. In the UI, click on Health & Monitoring -> Blockchain and scroll down to the Pending Transactions section and ensure that there are no pending transactions. We recommend that a node backup be performed when there are no transactions that are being processed by the node.

  • Restart the node to ensure all block state is committed to disk. For this, you may use either the API or UI:

    • Node Restart API Call
    • In the UI, click on the node on which you are performing a backup and click on Settings -> Restart node button. Wait until the node is in Started before proceeding.

Backup Node and Service Contents

After a backup configuration is created and associated with a node or a service, On-demand backup request can be triggered on the node or service via Kaleido API. Kaleido uploads a tar zipped file of the node or service's contents to the AWS S3 bucket when such a backup operation is triggered on the node or service. Please refer to the API documentation for node backup or API documentation for service backup or for details on the API request to trigger a backup operation.

If you created a AWS S3 Pre-Signed URL as the backup destination, backup the node content by providing the following detail in the API request. Backup using AWS S3 Pre-Signed URL is not yet available via Kaleido Console. Use the generated Pre-Signed URL as presigned_url request body field in the API request:

{
    "presigned_url": "https://test-bucket-1.s3.us-east-2.amazonaws.com/zzver79ou5-zzvt0b0ylf-quorum.tgz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AZIAJNQZWSKAX2VQHMMA%2F20191003%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20191003T161903Z&X-Amz-Expires=900&X-Amz-Signature=cf0f8ecfc701c921b6d324007006d43977b6a67ef0911bdefa8529a78a3683ce&X-Amz-SignedHeaders=host"
}