CloudFormation Datasource is a solution to make your CloudFormation templates more dynamic by allowing you to run API calls at deployment time. It emulates Terraform data sources behavior and allows you to retrieve AWS Organizations Id, the subnet with most free space or a specific KMS Key ARN from its alias.
It allows you to simplify some CloudFormation templates by removing parameters and mappings and rely instead on AWS API to retrieve the correct value at deployment time.
The solution is based on a generic custom resource backed with an AWS Lambda Function.
The custom resources takes the following inputs:
- AWS Service
- Parameters (as defined in the API Reference guide of the AWS Service)
- AWS Region
- Output selector (JMESPath expression)
The Lambda function parses the input and performs the API call on behalf of the CloudFormation template. It filters the result based on the output selector and return the value to the CloudFormation template. The value can then be used using the GetAtt intrinsic function.
Optionally, the function can lookup through CloudTrail events, the identity who perform the API call and check using AWS IAM policy simulator if it is allowed to perform the API call.
By default, the solution is deployed with a limited permissions set using the Amazon Managed policy ViewOnlyAccess which grants permissions to view resources and basic metadata across all AWS services. You can change it to fit your needs.
The behavior of the AWS Lambda function really mimicks the AWS CLI behavior when you pass the –query argument.
This template has been built with AWS SAM CLI
In order to deploy the template, run the following commands:
How to use it
Create a custom resource in your Cloudformation template and the service token should be the Arn of the lambda function. The Arn is exported as CFNDATASOURCE, so you can use it simply by entering ServiceToken: !ImportValue CFNDATASOURCE
It requires the name of the Service (‘organizations’, ’ec2’…) and the Action to invoke (‘DescribeOrganization’…), the parameters as a dict object (per the API reference of the service) and the query to filter the result using a JMESPath expression.
The example below shows how to return the OrganizationId
I’ve found this solution useful for recurring use-cases when I needed to retrieve the Organization Id to secure an Amazon S3 Bucket, or to select the subnet with the maximum address space available. I’ve documented some examples in the git repository and I hope it can be useful to you too. If you have some good examples you want to share, please submit a pull request.