Log Streaming

The Ethereum logs for each node serve as a rich datasource for devops-centric tasks such as application troubleshooting and operational health monitoring. While the Kaleido /logs API provides a convenient endpoint to retrieve these logs, it requires a manual configuration against the consortia/environment/node resource IDs and must be constantly polled to ensure up-to-date streams.

As an alternative to this configuration-intensive approach, Kaleido offers the option to integrate nodes with AWS’ monitoring and management service, Cloudwatch, and directly stream realtime logs. A fully-encompassed monitoring service provides the ability to visualize Kaleido logs alongside existing resources and processes, and surfaces an aggregated trove of data that can lead to greater insights and application optimization. For example, your Cloudwatch service could be customized to trigger metric-based alarms and issue automated actions based on certain inflections. This centralized view of the core application and business processes helps lead to more informed decisions and increased efficiency.

Configuring an IAM Role

NOTE: Support for IAM user-based longterm credentials (the AWS cloud configuration structure offered on previous Kaleido releases) is being deprecated. For newly created AWS configurations, you must use the IAM role approach outlined below.

Log into the AWS console and navigate to the IAM Service.
IAM roles will be used to grant Kaleido access to your KMS instance storing the master encryption key. Refer to the AWS guide for more information on IAM roles.
Click the Roles tab in the IAM navigation panel and proceed to Create a New Role.
Select Another AWS Account as trusted entity and use Kaleido's account ID 271815176711 in the Account ID field.
Skip Permissions, Tags, and finish creating role by specifying a Role Name.

Creating a Cloudwatch Logging Group

Click the Services tab at the top of the console and select Cloudwatch under the Management Tools column.
Click the Logs tab in the Cloudwatch navigation panel.
Click the Actions dropdown and select Create log group.
Enter a name for the logging group, e.g. “kaleidologs”, and create the group.
Take note of the AWS region in the URL, e.g. us-east-2. You will need this information when you configure your opsmetric configuration on Kaleido.

Configuring Cloudwatch Logging Permissions

Follow the below steps to configure log streaming into an AWS Cloudwatch Group:

Log into the AWS console and navigate to the IAM Service
Kaleido requires certain permissions in order to successfully propagate logs to a Cloudwatch Group. These permissions will need to be applied as a custom policy against an IAM role for which Kaleido's account (Account ID 271815176711) is the service principal. Refer to this AWS guide for more information on IAM roles.
Click the Policies tab in the IAM navigation panel.
Click Create Policy at the top of the screen and choose Cloudwatch Logs as the targeted service for the policy
Click the List dropdown and apply DescribeLogGroups and DescribeLogStreams access
Click the Write dropdown and apply CreateLogGroup, CreateLogStream and PutLogEvents
Create a name for the policy and review it, ensuring that you have provided access to the five actions above. Click Create policy to finish.
Click the Roles tab in the navigation panel to attach the newly created custom policy to an existing role.
Click the Add Permissions button and select the Attach existing policies directly option
Enter you custom policy name into the search bar and select it. Click Next: Review
Click the custom policy hyperlink to see the JSON output of the applied permissions. Make sure you see the five access permissions defined in the JSON object. The object should resemble the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        },
    ]
}

Click Finish.

Generating Kaleido IDs

You will need to use the administrative API to generate your log streams configuration on Kaleido. The API response will return two unique strings - configuration_id & external_id_seed - that will be used to edit your IAM role's trust policy. You will need to assemble an external_id string with a syntax of ${configuration_id}-${external_id_seed}-external-id, where configuration_id is ID of the KMS configuration object and external_id_seed is a unique value generated by Kaleido. Both are returned in the API call to generate the KMS configuration object.

An example request to create an IAM role based opsmetric configuration in your Kaleido environment:

https://console.kaleido.io/api/v1/consortia/:consortium_id/environments/:environment_id/configurations
{
    "membership_id": "<membership id belonging to your organization>",
    "name": "role-based-opsmetric-config",
    "type": "opsmetric",
    "details": {
        "provider": "aws",
        "region": "us-east-2",
        "role_arn": "<arn for the role you created to give kaleido access>",
        "group": "<log group name, optional>"
    }   
}

Sample response

{
    "membership_id": "<membership id belonging to your organization>",
    "name": "role-based-opsmetric-config",
    "type": "opsmetric",
    "details": {
        "provider": "aws",
        "region": "us-east-2",
        "role_arn": "<arn for the role you created to give kaleido access>",
        "group": "<log group name, optional>",
        "external_id_seed": "<unique id generated by kaleido>"
    },
    "_id": "<configuration id>",
    "_revision": "0",
    "created_at": "2021-06-08T02:32:46.152Z",
    "environment_id": "<environment id>"
}

If "_id" in the above JSON response is u0jj4apxhw and "external_id_seed" is u0f5drkxrb, then the fully assembled external_id string would be u0jj4apxhw-u0f5drkxrb-external-id.

Editing your IAM Role's Trust Policy

You now need to edit the trust policy of the IAM role used in your opsmetric configuration to include the external_id string. You can do this using following steps

Log into the AWS console and navigate to your IAM service.
Next, click the Roles tab in the navigation panel.
Click the Trust Relationships button and then Edit trust relationship button.
Update the policy document to use the external_id string as the Condition value. The updated policy document resembles the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::271815176711:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<external id string here>"
        }
      }
    }
  ]
}

Again, please refer to the following AWS guide to understand in detail how to use IAM roles and why they are required.