Overview

The AWS S3 integration allows for data deliveries from Live Data to be shared via S3 for consumption by customers. This provides a secure system for sharing data for those customers that use AWS.

Live Data supports 3 methods of configuring an S3 integration - users can configure any number of these integrations to support the delivery of different datasets to different endpoints.

Use your AWS role to access data in a bucket owned by Live Data
Grant a Live Data AWS role direct access to a bucket in your AWS account
Create a cross-account AWS role in your account and permit Live Data to assume the role

All methods are fully supported by AWS and follow best practices for cross-account access to data in S3 buckets.

Configuration Instructions

Use your AWS role to access data in a bucket owned by Live Data

One of the simplest to configure, in this paradigm you simply provide Live Data your AWS role ARN and we grant that role access to a scoped folder in a bucket in our account. You can then access data in that bucket using your role.

We use S3 Access Points to provide your role access to a specific folder in our delivery bucket -- Access Points are simply an overlay on top of S3 for permissions. Once you've completed the configuration steps below, the UI will provide you the S3 URL and Access Point ARN.

Prerequisites

Before completing the steps below, create a role in your AWS account. The role requires S3 permissions so that it can access the bucket and objects in the bucket. You can decide how much to restrict the S3 permissions based on how you use this role. At the broadest level, you could use the built-in AWS policy called AmazonS3ReadOnlyAccess. The most granular policy would limit the role's access only to the Access Point and data within it. An example of this policy is below, in which you would need to replace the ARNs with the ARN provided to you in the UI following the steps below.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*",
                "s3-object-lambda:*"
            ],
            "Resource": [
                "arn:aws:s3:us-east-1:100811426269:accesspoint/o-8eba54ab-101", // REPLACE 
                "arn:aws:s3:us-east-1:100811426269:accesspoint/o-8eba54ab-101/object/*", // REPLACE
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetAccessPoint",
                "s3:ListAccessPoints"
            ],
            "Resource": "arn:aws:s3:us-east-1:100811426269:accesspoint/*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::*",
            "Condition": {
                "StringEquals": {
                    "s3:DataAccessPointAccount": "100811426269"
                }
            }
        }
    ]
}

Steps

Log into gotlivedata.com and click on the Integrations page
Find the AWS S3 card and press Configure
Select Live Data S3 Bucket
Provide a name for this integration (this can be anything)
Enter the ARN of the role in your AWS account that you will use to access the data
Press Save

Verification

To verify that you have access to the data, you can try a simple aws s3 ls command. Ensure you are acting as the role provided in the prior steps. Gather the Bucket Path value from the prior steps and run:

aws s3 ls <bucket path value>

If the command completes with no errors, your integration setup is completed.

If you get an error message containing Access Denied, please verify you are using the correct role and the role has either broad S3 access against all resources, or is using the granular policy above with the correct value for the access point ARN as provided in the UI called Access Point ARN.

If you get an error message containing The specified bucket does not existthen double-check the bucket name and path in your command and try again.

Grant a Live Data AWS role direct access to a bucket in your AWS account

With this approach, you create bucket policies to allow Live Data to write directly to a bucket in your account. This approach may be preferred in situations where you want to trigger automation when a file is created in your bucket.

Steps

In AWS, navigate to the bucket you want to grant Live Data access to
Click the Permissions tab and edit the bucket policy to allow the Live Data role to access the bucket as required. At a minimum, the Live Data role must have ListBucket permissions on the bucket and PutObjecton the objects in the bucket. See the example policy below, but also reference AWS documentation and your own security team/standards to ensure these permissions are set up as required. The Live Data role arn is: arn:aws:iam::100811426269:role/customer-delivery-handler
Log into gotlivedata.com and click on the Integrations page
Find the AWS S3 card and press Configure
Select Your S3 Bucket
Select Grant Access to Live Data Role
Provide a name for this integration (this can be anything
Enter the bucket identifier for the bucket in your account. This can be the S3 URI, the ARN, or simply the bucket name.
Press Save

Sample bucket policy

{
	"Version": "2012-10-17",
	"Statement": [
        {
            "Sid": "LiveDataBucket",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::100811426269:role/customer-delivery-handler"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<your-bucket-name>"
        },
        {
            "Sid": "LiveDataObject",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::100811426269:role/customer-delivery-handler"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::<your-bucket-name>/*"
        }
    ]
}

Create a cross-account AWS role in your account and permit Live Data to assume the role

This approach uses the AWS cross-account role paradigm to allow Live Data access to your account. The most complex approach of the supported methods, it may be preferred under higher security standards.

Note: although an External ID is optional in AWS, it is required for the Live Data integration

Steps

In AWS create the cross-account role using the Live Data account ID 730335578765. When creating the cross-account role, make sure to create and note the External ID value for added security.
Grant the cross-account role access to the S3 bucket as desired. The Sample bucket policy below can be used as a reference for the minimum required permissions.
Log into gotlivedata.com and click on the Integrations page
Find the AWS S3 card and press Configure
Select Your S3 Bucket
Select Create a Cross-Account Role for Live Data to Use
Provide a name for this integration (this can be anything)
Enter the bucket identifier for the bucket in your account. This can be the S3 URI, the ARN, or simply the bucket name.
Enter the role ARN for the cross-account role that you created for this purpose
Enter the external ID value that you noted when creating the cross-account role
Press Save

Sample bucket policy

{
	"Version": "2012-10-17",
	"Statement": [
        {
            "Sid": "LiveDataBucket",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<your-bucket-name>"
        },
        {
            "Sid": "LiveDataObject",
            "Effect": "Allow",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::<your-bucket-name>/*"
        }
    ]
}