Question

What permissions and roles are required to set up Amazon Kinesis with Segment?

Product

Twilio Segment

Environment

Segment Console

Answer

To allow Segment to write data to your Amazon Kinesis Stream, you'll need to configure both an IAM policy and an IAM role in AWS. Below are the required steps:

Create an IAM policy

1. Sign in to the Identity and Access Management (IAM) console.
2. Follow these instructions to Create an IAM policy to allow Segment permission to write to your Kinesis Stream.
3. Select the Create Policy from JSON option and use the following template policy in the Policy Document field. Be sure to change the {region}, {account-id} and {stream-name} with the applicable values.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:PutRecord",
                "kinesis:PutRecords",
                "iam:SimulatePrincipalPolicy"
            ],
            "Resource": [
                "arn:aws:kinesis:{region}:{account-id}:stream/{stream-name}",
                "arn:aws:iam::{account-id}:role/{role-name}"
            ]
        }
    ]
}

NOTE: A previous version of this policy document only granted PutRecord access, which could slow down Kinesis write times by disallowing file batching. Substitute the updated policy document above to grant Kinesis PutRecords (plural) and allow batching. We’ve also requested iam:SimulatePrincipalPolicy, which will allow us to verify that the IAM Role has the appropriate Kinesis permissions without invoking the Kinesis API.

Create an IAM role

1. Follow these instructions to Create an IAM role to allow Segment permission to write to your Kinesis Stream.
2. When prompted to enter an Account ID, enter 595280932656. Make sure to enable Require External ID and enter your Segment Source ID as the External ID. This can be found by navigating to Settings > API Keys from your Segment source homepage.
   • NOTE: If you have multiple sources using Kinesis, enter one of their source IDs here for now and then follow the procedure outlined in the Multiple Sources section once you’ve completed this step and saved your IAM role.
3. When adding permissions to your new role, find the policy you created in step above and attach it.

Additional Information 

For step-by-step instructions on setting this up, see the documentation here.