API 101 Tutorial
It’s really important to us that everything you are able to do in the Kaleido console, you are also able to do through our convenient REST API. This content walks you through a series of high level calls and demonstrates the end-to-end process for creating a consortium and underlying environment via the command line. Note that blockchain-specific tasks (e.g. contract deployment) are beyond the scope of this tutorial; it is solely focused on resource generation.
Ready? Let’s give it a go...
Get your API key
You will need to briefly visit the UI in order to generate your API key. This key serves as the authorization token and allows you to send privileged administrative calls for your organization to the Kaleido backend.
Just navigate to the >Kaleido Console and once you’ve logged in, click the API tab at the top of the screen and then select + New API Key button to generate your key.
Make sure you note it down before closing the pop-up (we don’t store your API keys).
Bash (Linux or Mac) to generate common Authorization
and Content-Type
headers. Replace the YOUR_API_KEY
placeholder text with the key you just generated:
export APIURL="https://console.kaleido.io/api/v1"
export APIKEY="YOUR_API_KEY"
export HDR_AUTH="Authorization: Bearer $APIKEY"
export HDR_CT="Content-Type: application/json"
If you wish to host your resources in the EU or Asia Pacific, enumerate the region in your $APIURL
variable. The ap
qualifier resolves to Sydney, while ko
resolves to Seoul. For example:
export APIURL="https://console-eu.kaleido.io/api/v1"
export APIURL="https://console-ap.kaleido.io/api/v1"
export APIURL="https://console-ko.kaleido.io/api/v1"
Note: The forthcoming REST calls use
jq
to process the JSON output as well ascurl
.
JSON Syntax
In all sample curl
commands below, we have split the example across multiple lines for easier reading. If you want to put JSON on a single line, you will need to:
- remove the end-of-line backslashes
- use double quotes for values and fields within the body AND escape with single quotes.
- For example:
-d '{"name":"value"}'
If you need $VARIABLES
in the JSON, you will need to:
- use ONLY double quotes
- For example:
-d "{\"name\": \"$VARIABLE\"}"
If you see {"errorMessage": "Unexpected token in JSON"}
in response to a call, it is likely you have a problem with the quotes.
If you see [globbing] unmatched close brace/bracket
in response to a call, it is likely you have a misplaced or missing backslash.
If you see Invalid numeric literal at EOF
in response to a call, it is likely you have a missing header or unqualified variable in your path.
Create a new business consortium
Let’s go ahead and create a consortium.
Multi-line:
curl -H "$HDR_AUTH" -H "$HDR_CT" -s -d "{ \
\"name\": \"api101\", \
\"description\": \"Automation is great\" \
}" \
"$APIURL/consortia" | jq
Single-line:
curl -H "$HDR_AUTH" -H "$HDR_CT" -d '{"name":"api101”, "description":"Automation is Great"}' "$APIURL/consortia" | jq
Example output:
{
"name": "api101",
"description": "Automation is great",
"owner": "zzmutk03fu",
"_id": "zzrog1h91c",
"state": "setup",
"_revision": "0",
"created_at": "2018-05-09T12:24:57.337Z"
}
Create an environment
Next we need to create an environment associated with this consortium. A business consortium will likely have multiple environments for development, staging, production, etc… For the sake of this tutorial, we’ll just create a single instance.
An environment requires two pieces of configuration – provider
and consensus_type
. These fields are specified in the body of the call when posting to the /environments
endpoint. The provider
field declares the client type, with quorum
and geth
available as valid options. The consensus_type
field declares the consensus protocol, with raft
or ibft
available as compatible flavors with quorum
, and poa
mandated as the only choice for geth
. You also have the option of labeling your environment by passing a value to the name
field. While a name is not required, it’s recommended as a best practice in order to easily distinguish between multiple environments.
Let’s spin up a simple environment running Geth + PoA and name it Sample Environment
. The first line of the call sets the CONSORTIUM environment variable to the ID of our newly created api101
consortium. We then POST
to the /environments
endpoint and create a domain within the context of the specified consortium.
CONSORTIUM=$(curl -H "$HDR_AUTH" -H "$HDR_CT" -s "$APIURL/consortia?name=api101" | jq -r ".[0]._id")
curl -H "$HDR_AUTH" -H "$HDR_CT" -s -d "{ \
\"name\": \"Sample Environment\", \
\"provider\": \"geth\", \
\"consensus_type\": \"poa\" \
}" "$APIURL/consortia/$CONSORTIUM/environments" | jq
Example output:
{
"name": "Sample Environment",
"provider": "geth",
"consensus_type": "poa",
"initial_delay": 24,
"idle_hours": 24,
"_id": "zzfslxljb3",
"state": "initializing",
"enable_tether": false,
"block_period": 5,
"chain_id": 3288722771,
"node_list": [],
"region": "us-east",
"release_id": "zzc64zkexi",
"_revision": "0",
"created_at": "2018-05-09T12:59:02.078Z"
}
You may notice that there are additional parameters related to the environment (e.g. enable_tether
and block_period
), however those are beyond the scope of this high level tutorial.
Add members to your consortium<
Each business in the consortium has a membership that owns nodes and authorization credentials in the environments.
Now let’s create a couple of simulated participant members – Org1
& Org2
. The call is POST
ing to the /memberships
endpoint within the context of the api101
consortium and passing the org names as values to the required org_name
field.
Hit enter twice to kick off the second call:
# note that this sample command is setting the $CONSORTIUM env variable to the
# "api101" consortium id. If you want to target a different consortium, replace
# the api101 identifier or manually set the $CONSORTIUM variable
CONSORTIUM=$(curl -H "$HDR_AUTH" -H "$HDR_CT" -s "$APIURL/consortia?name=api101" | jq -r ".[0]._id")
# First member - Org1
curl -H "$HDR_AUTH" -H "$HDR_CT" -s -d "{ \
\"org_name\": \"Org1\" \
}" "$APIURL/consortia/$CONSORTIUM/memberships" | jq
# Second member - Org2
curl -H "$HDR_AUTH" -H "$HDR_CT" -s -d "{ \
\"org_name\": \"Org2\" \
}" "$APIURL/consortia/$CONSORTIUM/memberships" | jq
Example output:
{
"org_name": "Org1",
"org_id": "zzmutk03fu",
"state": "active",
"_id": "zzzkfkk8yz",
"minimum_nodes": 1,
"maximum_nodes": 1,
"_revision": "0",
"created_at": "2018-05-09T15:25:32.912Z"
}
{
"org_name": "Org2",
"org_id": "zzmutk03fu",
"state": "active",
"_id": "zzvqp5afzn",
"minimum_nodes": 1,
"maximum_nodes": 1,
"_revision": "0",
"created_at": "2018-05-09T15:25:39.500Z"
}
At this point we have a sample consortium – api101
– with ourselves as the default founding organization, as well as two simulated members – Org1
& Org2
. Additionally, we have an empty environment – Sample Environment
– within the consortium.
At any point you can exercise the GET
method to list out resources related to your Kaleido Organization. The API Key that is being passed in the authorization header scopes all calls to your specific org. For example, to see all consortia associated with your org:
To see all memberships within a specific consortium:
To see all environments within a specific consortium:
Create nodes
Finally to have a usable environment, we need to create the runtime nodes that will maintain the ledger and accept connections from applications. All nodes are created within the context of an environment and on behalf of a member participant in the consortium. As a result, the calls below are setting environment variables for the environment ID as well as membership IDs. The environment ID is declared in the path and the membership ID is passed in the body as a value to the required membership_id
field. As with an environment, the name field is optional, however, it is similarly recommended as a best practice.
Hit enter twice to kick off the second call:
# set the environment variable for ENVIRONMENT. this call assumes a single environment
# within $CONSORTIUM. $ENVIRONMENT is where we will add our nodes
ENVIRONMENT=$(curl -H "$HDR_AUTH" -H "$HDR_CT" -s "$APIURL/consortia/$CONSORTIUM/environments" | jq -r ".[0]._id")
# set the value for $MEMBER1 to the membership id for "Org1" and create a node
# named "Org1Node". Add this node to $ENVIRONMENT
MEMBER1=$(curl -H "$HDR_AUTH" -H "$HDR_CT" -s "$APIURL/consortia/$CONSORTIUM/memberships?org_name=Org1" | jq -r ".[0]._id")
curl -H "$HDR_AUTH" -H "$HDR_CT" -s -d "{ \
\"membership_id\": \"$MEMBER1\", \
\"name\": \"Org1Node\" \
}" "$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes" | jq
# set the value for $MEMBER2 to the membership id for "Org2" and create a node
# named "Org2Node". Add this node to $ENVIRONMENT
MEMBER2=$(curl -H "$HDR_AUTH" -H "$HDR_CT" -s "$APIURL/consortia/$CONSORTIUM/memberships?org_name=Org2" | jq -r ".[0]._id")
curl -H "$HDR_AUTH" -H "$HDR_CT" -s -d "{ \
\"membership_id\": \"$MEMBER2\", \
\"name\": \"Org2Node\" \
}" "$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes" | jq
Example output:
{
"membership_id": "zzt42o90rp",
"name": "Org1Node",
"role": "validator",
"state": "initializing",
"provider": "geth",
"consensus_type": "poa",
"id": "zzte3nghkd",
"_revision": "0",
"created_at": "2018-05-09T16:02:29.304Z"
}
{
"membership_id": "zzt42o90rp",
"name": "Org2Node",
"role": "validator",
"state": "initializing",
"provider": "geth",
"consensus_type": "poa",
"_id": "zzta4qrvld",
"_revision": "0",
"created_at": "2018-05-09T16:02:29.304Z"
}
Wait for the nodes to be initialized
We need to wait for the nodes to initialize before querying the detailed status. It typically takes between 30-60 seconds for each node to spin up. The below calls will set environment variables for each of the newly created nodes and wait for them to leave the initializing state. Once the calls complete we can query the nodes for details.
# set the value for $NODE1 as the "Org1Node" node id and await initialization
NODE1=$(curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes?name=Org1Node" \
| jq -r ".[0]._id")
while [ $(curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes/$NODE1" \
| jq -r '.state') == 'initializing' ]; \
do sleep 1; echo "Waiting for $NODE1"; done
# set the value for $NODE2 as the "Org2Node" node id and await initialization
NODE2=$(curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes?name=Org2Node" \
| jq -r ".[0]._id")
while [ $(curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes/$NODE2" \
| jq -r '.state') == 'initializing' ]; \
do sleep 1; echo "Waiting for $NODE2"; done
View the node status
Issue the following call to download the status of Org1Node
:
curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes/$NODE1/status" \
| jq
Example output:
{
"id": "05203d6363baf1d750ac6368022f610c65e8a12b60863196f9bb8be0d58a0ccb7b164d2dd50d202041b2f5580e40c948fb0fba55f8c19e495a5688449edd62d5",
"geth": {
"public_address": "0xbe7f4a7c45b38408f281108c176201ae439d24b5",
"validators": [
"0x21615c0746f03adf9a2834518a684c87952ac8d3"
]
},
"user_accounts": [
"0xBB7c7a73b4690CcBfDc9f3d945941846d7Be387C"
],
"block_height": 260,
"urls": {
"rpc": "https://zzfslxljb3-zzl8vfbx5e-rpc.us-east-2.kaleido.io",
"wss": "wss://zzfslxljb3-zzl8vfbx5e-wss.us-east-2.kaleido.io"
}
}
We see some interesting information such as the node ID, Ethereum accounts, etc… The URLs are of particular interest, as these are the endpoints that an external client or application would target for blockchain-specific tasks. We’ll return to the URLs in a moment.
Fetch the node logs
The following call will fetch the last 50 lines of a particular node’s logs. The logging output can be retrieved by passing the consortium, environment and node IDs in the path and calling the /logs
endpoint. For example:
curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes/$NODE1/logs/geth?maxlines=50" \
| jq -r '.[]'
If using quorum
as the environmental node protocol, you can fetch the constellation logs by replacing the geth
identifier with constellation
. You can also adjust the maxlines
argument to return a length of your choosing. For example:
curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes/$NODE1/logs/constellation?maxlines=25" \
| jq -r '.[]'
Check out Downloading logs for a handy tool to download your node logs.
Generate credentials to connect your app
Now that we have a running node, let’s generate an authorization credential for our application to connect. The authorization credentials, or app creds in Kaleido parlance, are a mandatory security protocol that authenticate external connections to the network. Application credentials are scoped to a membership within the consortium and are only applicable to the environment within which they are created. The username
:password
pair can be generated by specifying the consortium and environment IDs in the path and POST
ing to the /appcreds
endpoint with the relevant membership ID passed as a value to membership_id
in the body. For example:
curl -H "$HDR_AUTH" -H "$HDR_CT" -s \
-X POST -d "{\"membership_id\":\"$MEMBER1\"}" \
"$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/appcreds" \
| jq
Example output:
{
"membership_id": "zzt42o90rp",
"auth_type": "basic_auth",
"id": "zzjs02ugh6",
"_revision": "0",
"username": "zzjs02ugh6",
"password": "-JygLPMBqCGx_lmhJCWHeSPWaX6cIVOiymRVPLnzC-U"
}
Repeat the demonstrated process to generate app credentials for additional members that own nodes in the environment.
Be sure to save the password(s) somewhere safe. They are not stored by the Kaleido backend and cannot be redisplayed.
Construct the full URL for your JSON/RPC calls
Now we can combine the above authentication credentials with the URL from the earlier node status query, and construct the full URL for making JSON/RPC calls. Below is an example of a fully qualified HTTP node endpoint with the following syntax – https://{username}:{password}@{node-rpc-url}
.
RPC
https://zzjs02ugh6:-JygLPMBqCGx_lmhJCWHeSPWaX6cIVOiymRVPLnzC-U@zzfslxljb3-zzl8vfbx5e-rpc.us-east-2.kaleido.io
Alternatively, you can choose to leverage the web socket endpoint by mirroring the same syntax.
WSS
wss://zzjs02ugh6:-JygLPMBqCGx_lmhJCWHeSPWaX6cIVOiymRVPLnzC-U@zzfslxljb3-zzl8vfbx5e-wss.us-east-2.kaleido.io
Resource deletion
The logical corollary to resource creation is resource deletion. To remove an environment, send a DELETE
to the /environments
endpoint and specify the consortium and environment IDs in the path. Deleting an environment will also delete the underlying nodes, and by extension the blockchain. See below for a sample environment deletion call:
curl -H "$HDR_AUTH" -H "$HDR_CT" -X DELETE "$APIURL/consortia/$CONSORTIUM/environments/zzfslxljb3" | jq
Example output:
{
"_id": "zzfslxljb3",
"consensus_type": "poa",
"name": "Sample Environment",
"description": "Sample Test Environment",
"state": "deleted",
"provider": "geth",
"node_list": [],
"region": "us-east",
"_revision": "1"
}
Notice that the state of the environment is changed from setup
or live
to deleted
. A query against environments may still reveal deleted instances for a brief period of time, however these namespaces will no longer be accessible.
Use the DELETE
method and follow the same approach to remove other resources under your organization’s control. Deletion requires no data in the body of the call; you simply need to specify the relevant resource ID in the path.
Resource limitations & transactions
Please refer to the Kaleido Resource Model topic for a detailed listing on organizational and environment-specific resource limitations. At a high level, each organization is limited to two consortia and three environments per consortium. Each environment can host up to four member nodes.
REST driver
Now that you’re familiar with basics of the Kaleido API, here’s a handy python script that will drive each of the previously demonstrated REST calls. This proves particularly useful when aiming to quickly bootstrap a multi-member network, rather than manually exercising each API.
Requires
python3
to run
- Copy and save as
create-consortium.py
on your computer - Type
python3 create-consortium.py -h
for help
usage: consortia.py [-h] [--apikey APIKEY] [--apiurl APIURL] [--name [NAME]]
[--provider [{quorum}]] [--consensus [{raft,ibft}]]
[--environment [ENVIRONMENT]] [--waitok] [--verbose]
[--chainid [CHAINID]] [MEMBER [MEMBER ...]]
Create a consortium.
positional arguments:
MEMBER a list of member names
optional arguments:
-h, --help show this help message and exit
--apikey APIKEY, -k APIKEY
the API Key (or APIKEY env var)
--apiurl APIURL, -u APIURL
the API URL (or APIURL env var)
--name [NAME], -n [NAME]
the consortium name
--provider [{geth,quorum}], -p [{geth,quorum}]
the provider
--consensus [{poa,raft,ibft}], -c [{poa,raft,ibft}]
the consensus algorithm
--environment [ENVIRONMENT], -e [ENVIRONMENT]
the environment name
--waitok, -w wait to confirm each request
--verbose, -v verbose output
--chainid [CHAINID], -i [CHAINID]
a custom chain ID (such as 1)
- Set up your environment, for example:
export APIKEY=vjdymjbu-XxaFk3aR72eLeNXp9/J4lvOH7fdpQzFvud/rEKdIxPc=
export APIURL=https://console.kaleido.io/api/v1
- Interactively generate a consortium with verbose output and confirmation of each REST call:
- Script creation of a consortia with a single member (with verbose output):
Example python script
#!/usr/local/bin/python3
# Example script to create a single-account consortium, with an environment
# and a set of members. Each member is given one node, and the script
# waits until the nodes are initialized.
import urllib.request, json, os, re, argparse, time
providers = ['geth','quorum']
consensus_types = ['poa','raft','ibft']
parser = argparse.ArgumentParser(description='Create a consortium.')
parser.add_argument('--apikey', '-k', type=str,
required=(not 'APIKEY' in os.environ),
help='the API Key (or APIKEY env var)')
parser.add_argument('--apiurl', '-u', type=str,
required=(not 'APIURL' in os.environ),
help='the API URL (or APIURL env var)')
parser.add_argument('--name', '-n', type=str, nargs='?',
help='the consortium name')
parser.add_argument('--provider', '-p', type=str, nargs='?',
help='the provider', choices=providers)
parser.add_argument('--consensus', '-c', type=str, nargs='?',
help='the consensus algorithm', choices=consensus_types)
parser.add_argument('--environment', '-e', type=str, nargs='?',
help='the environment name')
parser.add_argument('--waitok', '-w', action='store_const', const=True,
help='wait to confirm each request')
parser.add_argument('--verbose', '-v', action='store_const', const=True,
help='verbose output')
parser.add_argument('--chainid', '-i', type=str, nargs='?',
help='a custom chain ID (such as 1)')
parser.add_argument('members', metavar='MEMBER', type=str,
nargs='*',
help='a list of member names')
args = parser.parse_args()
if (args.apikey is None):
args.apikey = os.environ['APIKEY']
if (args.apiurl is None):
args.apiurl = os.environ['APIURL']
headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer {0}'.format(args.apikey) }
input_queried = False
print_output = (args.verbose or args.waitok)
def enter_string(prompt):
in_string = ''
while len(in_string) == 0:
in_string = input('{0}: '.format(prompt))
input_queried = True
return in_string
def enter_strings(prompt):
out_arr = []
in_string = ''; end_prompt = ''
while len(in_string) > 0 or len(out_arr) == 0:
in_string = input('{0} {1}{2}: '.format(prompt, len(out_arr), end_prompt))
if (len(in_string) > 0):
out_arr.append(in_string)
end_prompt = ' (blank to finish)'
return out_arr
def pick_from_list(prompt, arr):
idx = ''
for (i, entry) in enumerate(arr):
print('{0}) {1}'.format(i, entry))
while not (re.match("^\d+$", idx) and int(idx) < len(arr)):
idx = input('{0}: '.format(prompt))
input_queried = True
return arr[int(idx)]
def call_api(method, path, json_data):
url = '{0}/{1}'.format(args.apiurl, path)
if print_output: print('--> {0} {1}'.format(method, url))
data_str = None
if (json_data != None):
data_str = json.dumps(json_data, indent=2)
if print_output: print(data_str)
req = urllib.request.Request(url, data=data_str.encode(), headers=headers, method=method)
if (args.waitok):
input('OK?')
with urllib.request.urlopen(req) as res:
status = res.getcode()
json_res = json.load(res)
if print_output:
print('<-- {0}'.format(res.getcode()))
print(json.dumps(json_res, indent=2))
return json_res
if (args.name is None):
args.name = enter_string('Enter a name for the consortium')
if (args.provider is None):
args.provider = pick_from_list('Choose the provider', providers)
if (args.consensus is None):
args.consensus = pick_from_list('Choose the consensus algorithm', consensus_types)
if (args.name is None):
args.name = enter_string('Enter a name for the consortium')
if (len(args.members) == 0):
args.members = enter_strings('Member name')
# Create the consortia
consortia_json = call_api('POST', 'consortia', { \
'name': args.name \
})
consortia_id = consortia_json[u'_id']
# Create the members
member_ids = []
for member_name in args.members:
member_json = call_api('POST', 'consortia/{0}/memberships'.format(consortia_id), { \
'org_name': member_name \
})
member_ids.append(member_json[u'_id'])
# Create the environment
env_json = call_api('POST', 'consortia/{0}/environments'.format(consortia_id), { \
'name': args.environment, \
'provider': args.provider, \
'consensus_type': args.consensus, \
'chain_id': int(args.chainid) if args.chainid else None })
environment_id = env_json[u'_id']
# Generate app key credentials
for (i, member_id) in enumerate(member_ids):
appkey_json = call_api('POST', 'consortia/{0}/environments/{1}/appcreds' \
.format(consortia_id, environment_id), { \
'membership_id': member_id
})
print('Basic auth credential for member "{0}":'.format(args.members[i]))
print('User: {0} Password: {1}'.format(appkey_json[u'username'], appkey_json[u'password']))
# Create the nodes
node_ids = []
for (i, member_id) in enumerate(member_ids):
node_json = call_api('POST', 'consortia/{0}/environments/{1}/nodes' \
.format(consortia_id, environment_id), { \
'name': "{0}'s Node".format(args.members[i]), \
'membership_id': member_id
})
node_ids.append(node_json[u'_id'])
# Wait for the nodes to be started
not_ready_nodes = node_ids.copy()
print('Waiting for nodes to start...')
while len(not_ready_nodes) > 0:
time.sleep(5)
new_not_ready_nodes = []
for (i, node_id) in enumerate(not_ready_nodes):
node_json = call_api('GET', 'consortia/{0}/environments/{1}/nodes/{2}' \
.format(consortia_id, environment_id, node_id), {})
node_state = node_json[u'state']
print('Node {0} is {1}'.format(node_id, node_state))
if (node_state != 'started'):
new_not_ready_nodes.append(node_id)
not_ready_nodes = new_not_ready_nodes
# Nodes are ready
print('All nodes ready. Details:')
print_output = True
for node_id in node_ids:
call_api('GET', 'consortia/{0}/environments/{1}/nodes/{2}/status' \
.format(consortia_id, environment_id, node_id), {})</code></pre>