Knowledge Base Home

Public and Private Transactions

This tutorial is modeled after a sample consortium of three organizations – SupplierShipperRetailer – and an underlying environment running a Quorum based blockchain using RAFT consensus. You do not need to create an identical scenario, however you will need at least three members (and by extension their underlying nodes) in order to recreate the principles demonstrated here. If you have not yet done so, visit the Kaleido Platform UI and construct your first consortium + environment before returning to this doc. To mimic the orchestration presented in this tutorial:

  • Use your Kaleido Organization as the proxy operator of the network
  • Add three member organizations – SupplierShipper and Retailer
  • Select Quorum as the protocol and Raft as the consensus algorithm for your environment configuration
  • Add a node for each of the member organizations – Supplier NodeShipper Node and Retailer Node

NOTE: It is highly recommended to read through the API 101 tutorial prior to continuing, as this content assumes a general understanding of resource IDs, node URLs, application credentials and basic functionality of the Kaleido REST API.

Environment

Before diving into smart contract deployment and transaction invocations, let’s take a look at our environment and identify the key parameters. We’ll be using the REST API to grab these details, however you can also discern the same information through the user interface. If you prefer to use the UI for node and environment information, jump down to the Application Credentials section and proceed with this tutorial.

# $APIKEY, $APIURL, $HDR_AUTH and $HDR_CT have all been set

curl -H "$HDR_AUTH" -H "$HDR_CT" "$APIURL/consortia" | jq

This returns us a JSON array with some high level descriptors of our consortium, including the consortia ID. The expected output should look similar to the following:

{
    "_id": "n3wjyz4k",
    "name": "Supply Chain",
    "description": "send goods from point A to point B",
    "mode": "single-org",
    "owner": "umnplw6j",
    "state": "setup",
    "_revision": "0",
    "created_at": "2018-04-02T19:13:37.021Z"
  }

NOTE: The values for you consortia, environment, etc. will be different than this sample. Be sure to set your environment variables or modify the REST calls accordingly.

Now we can set the consortium ID as a variable and then query our environment:

export CONSORTIUM="n3wjyz4k" && curl -H "$HDR_AUTH" -H "$HDR_CT" "$APIURL/consortia/$CONSORTIUM/environments" | jq

This time we receive back some more extensive output, including the environment ID and a list of the environment’s nodes. The expected output should look similar to the following:

{
    "_id": "vsy7gjc4",
    "name": "Supply Chain Environment",
    "description": "Initial Environment for Supply Chain",
    "state": "live",
    "region": "us-east",
    "provider": "quorum",
    "consensus_type": "raft",
    "node_list": [
      "fr4dabcd",
      "wvo0abcd",
      "amgpabcd",
      "e2p3abcd"
    ],
    "_revision": "0",
    "created_at": "2018-04-02T19:50:13.917Z"
  }

The main item of interest is the environment ID, so let’s go ahead and set that as a variable:

export ENVIRONMENT="vsy7gjc4"

Nodes

Great. Now we can leverage our two previously set environment variables and fetch the relevant particulars for each of our nodes:

curl -H "$HDR_AUTH" -H "$HDR_CT" "$APIURL/consortia/$CONSORTIUM/environments/$ENVIRONMENT/nodes" | jq

The expected output should look similar to the following:

{
    "_id": "fr4dabcd",
    "name": "Supplier Node",
    "membership_id": "b3rle140",
    "role": "validator",
    "state": "started",
    "provider": "quorum",
    "consensus_type": "raft",
    "_revision": "1",
    "created_at": "2018-04-02T19:50:31.849Z",
    "quorum_private_address": "pL2rj52FwAfVCtDxC2qCIogF7X9wkevMbG+GfgARk0I=",
    "url": "https://vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io",
    "ws_url": "wss://vsy7gjc4-fr4dabcd-wss.us-east-2.kaleido.io",
    "updated_at": "2018-04-02T19:50:55.281Z"
  },
  {
    "_id": "wvo0abcd",
    "role": "monitor",
    "name": "System Monitor",
    "membership_id": "sys--mon",
    "state": "started",
    "provider": "quorum",
    "consensus_type": "raft",
    "_revision": "1",
    "created_at": "2018-04-02T19:50:31.911Z",
    "quorum_private_address": "yBNo+aSIe5uAKk1/YynzP0BoTe4kim4SPBfA1U3VaVc=",
    "url": "https://vsy7gjc4-wvo0abcd-rpc.us-east-2.kaleido.io",
    "ws_url": "wss://vsy7gjc4-wvo0abcd-wss.us-east-2.kaleido.io",
    "updated_at": "2018-04-02T19:51:56.594Z"
  },
  {
    "_id": "amgpabcd",
    "name": "Shipper Node",
    "membership_id": "dawthqga",
    "role": "validator",
    "state": "started",
    "provider": "quorum",
    "consensus_type": "raft",
    "_revision": "1",
    "created_at": "2018-04-02T19:50:42.765Z",
    "quorum_private_address": "eRGS9qUizlV+JB6OdinyAHSJSvd1jCFWPHNZYoatilc=",
    "url": "https://vsy7gjc4-amgpabcd-rpc.us-east-2.kaleido.io",
    "ws_url": "wss://vsy7gjc4-amgpabcd-wss.us-east-2.kaleido.io",
    "updated_at": "2018-04-02T19:51:52.428Z"
  },
  {
    "_id": "e2p3abcd",
    "name": "Retailer Node",
    "membership_id": "itopkgke",
    "role": "validator",
    "state": "started",
    "provider": "quorum",
    "consensus_type": "raft",
    "_revision": "1",
    "created_at": "2018-04-02T19:50:51.164Z",
    "quorum_private_address": "L1wQmDvFV3bjYXa0NrFiBJRrSd+FUQxxp0TfYzm1wlQ=",
    "url": "https://vsy7gjc4-e2p3abcd-rpc.us-east-2.kaleido.io",
    "ws_url": "wss://vsy7gjc4-e2p3abcd-wss.us-east-2.kaleido.io",
    "updated_at": "2018-04-02T19:51:48.904Z"
  }

Yes lot’s of output, but don’t be alarmed. We’re only concerned with three fields – "url""ws_url"and "quorum_private_address". For the sake of readability we will print these out below:

{
    "name": "Supplier Node",
    "quorum_private_address": "pL2rj52FwAfVCtDxC2qCIogF7X9wkevMbG+GfgARk0I=",
    "url": "https://vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io",
    "ws_url": "wss://vsy7gjc4-fr4dabcd-wss.us-east-2.kaleido.io",


    "name": "Shipper Node",
    "quorum_private_address": "eRGS9qUizlV+JB6OdinyAHSJSvd1jCFWPHNZYoatilc=",
    "url": "https://vsy7gjc4-amgpabcd-rpc.us-east-2.kaleido.io",
    "ws_url": "wss://vsy7gjc4-amgpabcd-wss.us-east-2.kaleido.io",

    "name": "Retailer Node",
    "quorum_private_address": "L1wQmDvFV3bjYXa0NrFiBJRrSd+FUQxxp0TfYzm1wlQ=",
    "url": "https://vsy7gjc4-e2p3abcd-rpc.us-east-2.kaleido.io",
    "ws_url": "wss://vsy7gjc4-e2p3abcd-wss.us-east-2.kaleido.io",
  }

Constellation

Before continuing, let’s go ahead and set the private address variable for each of our nodes. If you’re unfamiliar with the quorum client and its corresponding “constellation” module, or simply want to refresh your knowledge, take a look at documentation surrounding private transactions.

As the "quorum_private_address" value is truly the public key for the constellation’s transaction manager, we’ll use a TM demarcation when constructing these variables:

export SUPPLIER_TM="pL2rj52FwAfVCtDxC2qCIogF7X9wkevMbG+GfgARk0I="
export SHIPPER_TM="eRGS9qUizlV+JB6OdinyAHSJSvd1jCFWPHNZYoatilc="
export RETAILER_TM="L1wQmDvFV3bjYXa0NrFiBJRrSd+FUQxxp0TfYzm1wlQ="

These variables will resurface at the end of the tutorial. For the time being it suffices to say that these are the values we will supply when we want to target a specific node for a private transaction.

Application Credentials

Our final piece of housekeeping is to generate some basic auth credentials for each of the member organizations. These credentials are especially important, as they are the mechanism that secures external access (e.g. an application) to the node’s runtime. Without a valid app credential, any blockchain-specific actions (i.e. query/invoke) will be rejected when the connection is attempted. These credentials are mandatory regardless of transaction classes.

Navigate into your environment within the Kaleido dashboard and click the Add dropdown at the top right of the screen. Select the New App Credentials option. Go ahead and generate the basic auth credentials for each of your members, remembering to save the password each time (it cannot be redisplayed). In our case we’ll have three total; one apiece for the Supplier, Shipper and Retailer.  The syntax is username:password:

  • Supplier Basic Auth – abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U
  • Shipper Basic Auth – abcd5ydo:F2baITcWbY6t16cBKPQjUVy0ZczNs-xHo5rzZaptklQ
  • Retailer Basic Auth – abcdn3qj:byMEM357FFhnZV3phed_LJAYwDYVTsr3kNN80om5HHw

With the basic auth credentials now in hand, we can bundle them accordingly with each member node’s URL. First, let’s have a look at the two pieces that will comprise the fully fledged URL required by the Kaleido Platform:

  • Supplier Basic Auth – abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U
  • Supplier URL – https://vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io

These two strings combine to become:

https://abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U@vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io

This is the externally accessible or “fully-qualified” URL for the Supplier Organization’s node with the following syntax – https://username:password@rpcURL

Now that the composition is clear, we can go ahead and export an env variable for each external URL:

export SUPPLIER_URL="https://abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U@vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io"
export SHIPPER_URL="https://abcd5ydo:F2baITcWbY6t16cBKPQjUVy0ZczNs-xHo5rzZaptklQ@vsy7gjc4-amgpabcd-rpc.us-east-2.kaleido.io"
export RETAILER_URL="https://abcdn3qj:byMEM357FFhnZV3phed_LJAYwDYVTsr3kNN80om5HHw@vsy7gjc4-e2p3abcd-rpc.us-east-2.kaleido.io"

Websocket Provider

The web3 API also offers support for connections over the web socket protocol. To use web socket endpoints with basic auth, follow the demonstrated approach and simply use the "ws_url" value as your URL baseline.

And export env variables for your web socket endpoints:

export SUPPLIER_URL="wss://abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U@vsy7gjc4-fr4dabcd-wss.stage.kaleido.io"
export SHIPPER_URL="wss://abcd5ydo:F2baITcWbY6t16cBKPQjUVy0ZczNs-xHo5rzZaptklQ@vsy7gjc4-amgpabcd-wss.stage.kaleido.io"
export RETAILER_URL="wss://abcdn3qj:byMEM357FFhnZV3phed_LJAYwDYVTsr3kNN80om5HHw@vsy7gjc4-e2p3abcd-wss.stage.kaleido.io"

Source Code

Create a local working directory for this sample and clone the kaleido-js github repository:

git clone https://github.com/kaleido-io/kaleido-js.git

Change into the deploy-transact directory where the program and smart contract live:

cd deploy-transact

Take a look at the directory, you should have the following structure:

Kaleido-MacBook-Pro:deploy-transact kaleido$ ls
README.md		simplestorage.sol	test.js   package.json

Using the WebsocketProvider Method

If you want to connect over web sockets, open the test.js program and edit the providers attribute to specify the WebsocketProvider method. The code should look as such:

console.log(`1. Connecting to Quorum node: ${url}`);
let web3 = new Web3(new Web3.providers.WebsocketProvider(url));

Now install the node_modules:

# If you receive a permissions error, try running the command as sudo.
npm install

The two key artifacts for this tutorial are the SimpleStorage solidity smart contract and the corresponding node program (test.js) that will drive calls to the network. SimpleStorage is as basic as a smart contract can get. It declares a state variable – storedData – of type uint (unsigned integer) and exposes a set function to modify the data, and a get function to retrieve the stored value. The node program is pre-baked with an init value of 10 which is set upon deployment of the smart contract. You can easily modify the initialization arguments, however for the sake of this tutorial we’ll leave it as is.

A Word on Quorum

The Quorum client is capable of supporting public transactions (identically to Geth), however it also offers the capability for private transactions through the specification of a privateFor field in the transaction payload. This flexibility is very powerful, as oftentimes there are multiple flavors of transactions within a overall business venture. As an example, our Supplier, Shipper and Retailer may all need to know the quantity of the merchandise being shipped, but the Shipper doesn’t necessarily need any information regarding the agreed upon price.

Public Transactions

Deploy the SimpleStorage smart contract with an initial value of “10” against an Ethereum account on the Supplier’s node:

node test.js --url=$SUPPLIER_URL --deploy --verbose

What’s Happening?

The program connects to the targeted node, derives the node’s Ethereum account using the web3 getAccountAPI, compiles the solidity code and generates the ABI, and finally deploys the bytecode with the specified arguments using the node’s default Ethereum account to sign the transaction object. Upon successful deployment, the program returns the smart contract address for subsequent reads and writes. The expected output should look similar to the following:

1. Connecting to Quorum node: https://abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U@vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io
	Found account in the target node: 0x1338cA30d67f9B67a5DAD8CD97f2200cF3e35E56
2. Deploying smart contract
  blockHash: '0x5c6b2466d17ec4c895fe28e958655c174980e491b99965ace592df1ce8997d43',
  blockNumber: 1,
  contractAddress: '0xEA057dC689605d41ceD62aa4aECb813796450265',
  cumulativeGasUsed: 132684,
  from: '0x1338ca30d67f9b67a5dad8cd97f2200cf3e35e56',
  gasUsed: 132684,
  logsBloom: '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
  status: '0x1',
  to: null,
  transactionHash: '0x94a2affbb70bbf28bfdbe9570f01f106c6ca9967d9a452e5dedc3ef1e017a8c7',
  transactionIndex: 0,
  events: {}
	Smart contract deployed, ready to take calls at "0xEA057dC689605d41ceD62aa4aECb813796450265"

We can see that an account – 0x1338cA30d67f9B67a5DAD8CD97f2200cF3e35E56 – was discovered in the Supplier’s node and that the contract is now callable at – 0xEA057dC689605d41ceD62aa4aECb813796450265. Since we forwent the privacy features offered by the constellation, this deployment simply updated the public state trie for each of the nodes in our environment. As a result, a query on this smart contract by any node should reveal the init value specified in the deploy arguments (10 in our case). Go ahead and query each of the nodes. For example:

node test.js --url=$SHIPPER_URL --contract=0xEA057dC689605d41ceD62aa4aECb813796450265  --query

The expected output should look similar to the following:

1. Connecting to Quorum node: https://abcd5ydo:F2baITcWbY6t16cBKPQjUVy0ZczNs-xHo5rzZaptklQ@vsy7gjc4-amgpabcd-rpc.us-east-2.kaleido.io
2. Calling smart contract at "0xEA057dC689605d41ceD62aa4aECb813796450265" for current state value
	Smart contract current state: "10"

The node program also exposes the set function so that you can modify the state (i.e. the value) for the stored data. For example:

node test.js --url=$RETAILER_URL --contract=0xEA057dC689605d41ceD62aa4aECb813796450265 --set=12345

Now any queries against the smart contract will return “12345”.

Private Transactions

Let’s return to the scenario where we want to obscure some state data from the Shipper. In order to do so we’ll need to leverage the Constellation’s transaction manager and enclave sub-modules which exist in parallel with the Quorum client. A thorough document describing private transactions and quorum architecture already exists, so there’s no need to rehash the minutiae on transaction flow, encryption, etc. Our main concern from the external application side is how to target nodes within a private context. The answer is to use the private address (the transaction manager’s public key) for each of the nodes that need visibility to the private data. You’ll recall that earlier we set these values as environment variables, so we can easily retrieve the address for the Retailer node:

echo $RETAILER_TM
L1wQmDvFV3bjYXa0NrFiBJRrSd+FUQxxp0TfYzm1wlQ=

Now let’s deploy SimpleStorage as a private smart contract between the Supplier and the Retailer. Note that the target node’s private address does not need to be specified, only its endpoint:

node test.js --url=$SUPPLIER_URL --privateFor='["L1wQmDvFV3bjYXa0NrFiBJRrSd+FUQxxp0TfYzm1wlQ="]' --deploy --verbose

Here’s the top portion of the expected output:

1. Connecting to Quorum node: https://abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U@vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io
	Found account in the target node: 0x1338cA30d67f9B67a5DAD8CD97f2200cF3e35E56
2. Deploying smart contract
  blockHash: '0xa0e0090810b625c040d547b7aa3eaac278b0e398da8c5ff7ecad7852fcb7b922',
  blockNumber: 3,
  contractAddress: '0xfAdDdC1A1Ec8c23f46053230539231973063c1cC',

You’ll notice that we used the same user (ethereum) account in the Supplier’s node, but this time we have a different smart contract address to call – 0xfAdDdC1A1Ec8c23f46053230539231973063c1cC – that will have state independent of our initial SimpleStorage deployment. First, let’s check and see if the Retailer node can retrieve any state for this contract. We’re expecting to see “10”:

node test.js --url=$RETAILER_URL --contract=0xfAdDdC1A1Ec8c23f46053230539231973063c1cC --query

Expected output:

1. Connecting to Quorum node: https://abcdn3qj:byMEM357FFhnZV3phed_LJAYwDYVTsr3kNN80om5HHw@vsy7gjc4-e2p3abcd-rpc.us-east-2.kaleido.io
2. Calling smart contract at "0xfAdDdC1A1Ec8c23f46053230539231973063c1cC" for current state value
	Smart contract current state: "10"

Finally let’s ensure that the Shipper Node has no visibility for this new contract:

node test.js --url=$SHIPPER_URL --contract=0xfAdDdC1A1Ec8c23f46053230539231973063c1cC --query

Expected output:

1. Connecting to Quorum node: https://abcd5ydo:F2baITcWbY6t16cBKPQjUVy0ZczNs-xHo5rzZaptklQ@vsy7gjc4-amgpabcd-rpc.us-east-2.kaleido.io
2. Calling smart contract at "0xfAdDdC1A1Ec8c23f46053230539231973063c1cC" for current state value
(node:27093) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Couldnt decode uint256 from ABI: 0x
(node:27093) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

The key snippet is Couldn't decode uint256 from ABI telling us that the node was unable to successfully interface with the smart contract bytecode. Or in other words, this node has no data in its private state database corresponding to this contract’s address.

Set a new private value

As with the public transaction, you can call the set function after deployment of the contract. Note though that you must still specify the privateFor field each time you want to update state. For example:

node test.js --url=$SUPPLIER_URL --contract=0xfAdDdC1A1Ec8c23f46053230539231973063c1cC --privateFor='["L1wQmDvFV3bjYXa0NrFiBJRrSd+FUQxxp0TfYzm1wlQ="]' --set=99999

Externally Signed Transactions

It is also possible to use an external account outside of the target node to sign transactions. The sign function within the program looks for a local web3 keystore at ~/.web3keystore and uses the key material within that account to sign.  If a local account is not found on your machine, the function calls the web3.eth.accounts.create API to generate a local account, sign the transaction with the account’s private key and then submit the signed object through the web3.eth.sendSignedTransaction API. For example:

node test.js --sign --url=$SUPPLIER_URL --contract=0xfAdDdC1A1Ec8c23f46053230539231973063c1cC --set=4321

Expected output:

1. Connecting to target node: https://abcdfxkd:nAucqJ1HywuSLY8O7t4qSS6fMLOXX5a3mX4VmhvnQ-U@vsy7gjc4-fr4dabcd-rpc.us-east-2.kaleido.io

Signed payload: 0xf88480808307a120949fd3ab9853a23d5855d60edeae98861ff274defc80a460fe47b100000000000000000000000000000000000000000000000000000000000030391ca02db3c86f679c288746b0217c093111a961a07c3a4965f278b3ae6036dbe28d93a07fe4fd9820e5cda5586d43ac955c014be2eace6bb0643f7f5489cb06829d3bbb
	
            Set new value to 4321

DONE!

Summary

This tutorial demonstrates the relationship between application credentials and a member node’s URL, and exposes the appropriate steps to construct the fully fledged endpoint such that external connections are properly secured. With a properly formed URL in hand, nodes are targetable by two classes of transactions – public & private. Public transactions follow the same flow as core Geth, where smart contracts are deployed and subsequent transactions calling the smart contract’s address result in identical state information being replicated into the public state trie of all nodes in the environment. Private transactions make use of the Quorum node’s constellation module, and allow a subset of participants (specified through the privateFor field in the transaction payload) to update state in their private state trie. Nodes not identified as privateFor recipients will be unable to retrieve state information for private contracts and their ensuing transactions.

Prev Known Clients Next Connecting with Truffle