Doc Store Intro

Kaleido Document Store

Use the document store service to securely upload and transfer files within a Kaleido environment. With blockchains being poorly suited for large payloads and sensitive data such as PII, an ancillary file sharing utility that can exist next to the chain provides immense benefit at both a business and security level. Transactions simply need to reference a file’s hash, rather than dealing with the file contents directly.

Any uploaded or received file can be conveniently organized using intuitive folder structures, and persisted in either Kaleido hosted storage or cloud-delegated and user-controlled services such as AWS S3 Buckets or Azure Blob containers. All transferred data is deterministically hashed, signed, compressed and asymmetrically encrypted in flight using proven public key infrastructure techniques, offering provable guarantees that only the intended recipient can effectively decrypt the packet. The service leverages the On-Chain Registry service for address look up and certificate functionality, as well as the Kaleido network’s Kafka backbone for high throughput, fault tolerant reliable delivery and transport. Interaction with service is flexible, with support for the in-console graphical user interface, Kaleido RESTful APIs and socket.io connections for reliable event notification.

Configuring and Provisioning your Document Store

Document stores are member-specific services and are only accessible by the owning organization.  Use the Kaleido console as the recommended happy path to configure and deploy an instance of the Document Store Service.  If you wish to use the Kaleido API, skip to the bottom of the page for instructions on programmatically configuring and provisioning the service.

Via the console

  • Navigate to an existing environment and click the + ADD dropdown in the top portion right of the screen and click Add Services.
  • Select the Document Store Service from the catalog and click ADD.
  • Optionally supply an arbitrary name for the service.
  • Lastly, choose the location for your storage bucket.  The choices are Kaleido hosted, AWS or Microsoft Azure.

Kaleido storage

If you wish to use Kaleido storage, leave the selector on Kaleido and click ADD deploy the service.  This will result in your files being stored on a Kaleido-hosted container.

Provision Document Store

Click ADD to finish and provision the service.

AWS S3 storage

If you wish to delegate your file storage to an Amazon S3 bucket, switch the selector to AWS and click NEXT.

AWS Backed storage

You will be redirected to a configuration modal where the required parameters mirror those of an S3 backup integration with a Kaleido node.  The instructions for properly generating storage buckets, user permissions and custom policies can be found here.  Below is an example of the configuration screen for Amazon S3 delegated storage:

Configure AWS S3 Bucket

The configuration parameters are defined as follows:

  • NAME - arbitrary string for the configuration object
  • REGION STRING - The region within AWS containing the S3 bucket. For example, us-east-2.
  • API KEY - User ID of the IAM user or role that has been applied with S3 write permissions
  • API SECRET - Secret of the IAM user or role that has been applied with S3 write permissions
  • S3 BUCKET NAME - Name of the S3 bucket to be used for the Document Store Service

Press CONTINUE to finalize and provision the service.  If the AWS configuration parameters were properly supplied you will see a success modal displayed.

Docstore Success

Azure Blob storage

If you wish to delegate your file storage to an Azure Blob container, switch the selector to Azure and click NEXT.

Azure Backed storage

You will be redirected to a configuration modal displaying the required parameters for Azure storage.  Below is an example of the configuration screen for Azure Blob delegated storage:

Configure Azure Storage Account

The configuration parameters are defined as follows:

  • NAME - arbitrary string for the configuration object.
  • STORAGE ACCOUNT NAME - The name of your Azure Storage Account.
  • ACCOUNT ACCESS KEY - The access key for your Azure Storage Account.  Note that this key must be ascribed as the signing key in the "Shared Access Signature" settings within your Storage Account.
  • CONTAINER NAME - Name of the blob container to be used for the Document Store Service.

Press CONTINUE to finalize and provision the service.  If the Azure configuration parameters were properly supplied you will see a success modal displayed.

Docstore Success

Deployed Instances

Instances of the Document Store Service will appear on your environment panel below MEMBER SERVICES.  Access the service by clicking the name or selecting the ellipsis dropdown and clicking "View Dashboard."

Via the API

NOTE: The following deployment approach assumes a strong understanding of the Kaleido APIs. Please refer to the Kaleido Resource Model for object relationships, the API 101 topic for sample CRUD operations and api.kaleido.io for detailed descriptions of the various endpoints and routes.

The following examples make use of cURL and assume that the following environment variables have been properly set:

export APIURL="https://console.kaleido.io/api/v1"
export APIKEY="YOUR_API_KEY"
export HDR_AUTH="Authorization: Bearer $APIKEY"
export HDR_CT="Content-Type: application/json"

If you are targeting an environment outside of the US or on Azure, make sure to modify your URL accordingly. The ap qualifier resolves to Sydney, while ko resolves to Seoul. The us1 qualifier designates an environment configured with Azure US West as the home region. For multi-region environments, the home region is determined by the location of the first deployed node:

export APIURL="https://console-eu.kaleido.io/api/v1"
export APIURL="https://console-ap.kaleido.io/api/v1"
export APIURL="https://console-ko.kaleido.io/api/v1"
export APIURL="https://console-us1.kaleido.io/api/v1"

Kaleido-hosted storage

Use the POST method against the /services endpoint specifying the consortia and environment IDs in the path, and supplying the membership ID, a service type of "documentstore" and an arbitrary name for the service in the body of the call.  For example:

# replace the placeholder values with your actual Kaleido resource IDs
curl -X POST -H "$HDR_AUTH" -H "$HDR_CT" "$APIURL/consortia/{consortia_id}/environments/{environment_id}/services" -d '{"membership_id":"{membership_id}", "service":"documentstore", "name":"Company ABC - documentstore"}' | jq

Cloud-delegated storage

If electing for cloud-delegated storage, two separate calls are required to provision and configure the service.  The first is a call to the /configurations endpoint.  This call ingests the configuration parameters - outlined in the corresponding cloud sections above - for your desired cloud storage (AWS or Azure) and returns a configuration ID.  The second call targets the /services endpoint and requires the previously generated configuration ID to be supplied within the body of the call.

Use the POST method to generate the storage configuration object for the service and optionally format the output using jq.  This example command uses Azure as the cloud provider.

# replace the placeholders with the actual values for your Kaleido and Azure resources
curl -X POST -H "$HDR_AUTH" -H "$HDR_CT" "$APIURL/consortia/{consortia_id}/environments/{environment_id}/configurations" -d '{"name":"exampleconfig", "type":"storage", "membership_id":"{membership_id}", "details":{"provider":"azure", "bucket":"CONTAINER_NAME", "user_id":"STORAGE_ACCOUNT", "user_secret":"ACCOUNT_ACCESS_KEY"}}' | jq

This will return you a JSON payload containing the service ID which will be used in the forthcoming call.

Next, use the POST method once again to provision the Document Store Service.  Note that the previously generated configuration ID is passed as a nested argument to the "details" parameter in the body of this call.

# replace the placeholders with the actual values for your Kaleido resources
curl -X POST -H "$HDR_AUTH" -H "$HDR_CT" "$APIURL/consortia/{consortia_id}/environments/{environment_id}/services" -d '{"name":"Company ABC - documentstore", "service":"documentstore", "membership_id":"{membership_id}", "details":{"storage_id":"{configuration_id}"}}' | jq

The provision call will return you the service details for your newly generated Document Store Service.  Lastly, you can optionally call a GET on the /services endpoint and specify the service id in the route to view the status of your service.  For example:

# replace the placeholders with the actual values for your Kaleido resources
curl -X GET -H "$HDR_AUTH" -H "$HDR_CT" "$APIURL/consortia/{consortia_id}/environments/{environment_id}/services/{service_id}" | jq