cfn-include
is a preprocessor for CloudFormation templates which extends CloudFormation's intrinsic functions.
For example, Fn::Include
provides a convenient way to include files, which can be local, a URL or on an S3 bucket (with proper IAM authentication if necessary). It supports both JSON and YAML as input and output format. CloudFormation's tag syntax for YAML (e.g. !GetAtt
) is supported as well.
cfn-include
tries to be minimally invasive, meaning that the template will still look and feel like an ordinary CloudFormation template. This is what sets cfn-include
apart from other CloudFormation preprocessors such as CFNDSL, StackFormation and AWSBoxen. There is no need to use a scripting language or adjust to new syntax. Check them out though, they might be a better fit for you.
Functions
Tag-based syntax is available in YAML templates. For example,Fn::Include
becomes !Include
.
You can either install cfn-include
or use a web service to compile templates.
npm install --global cfn-include
The web service can be called with your favorite CLI tool such as curl
.
curl https://api.netcubed.de/latest/template -XPOST -d @template.json
cfn-include <path> [options]
-
path
location of template. Either path to a local file, URL or file on an S3 bucket (e.g.
s3://bucket-name/example.template
)
Options:
-m, --minimize
minimize JSON output [false]-t, --validate
validate compiled template [false]-y, --yaml
output yaml instead of json [false]--version
print version and exit
curl https://api.netcubed.de/latest/template?[options] -XPOST -d @<path>
-
path
the contents of
path
will bePOST
ed to the web service. Seeman curl
for details.
Options:
Options are query parameters.
validate=false
do not validate template [true]
YAML
---
AWSTemplateFormatVersion: "2010-09-09"
Mappings:
Region2AMI:
!Include https://api.netcubed.de/latest/ami/lookup?architecture=HVM64
Resources:
Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: !FindInMap [ Region2AMI, !Ref AWS::Region, AMI ]
UserData:
Fn::Base64:
Fn::Sub:
!Include { type: string, location: userdata.sh }
JSON
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Mappings": {
"Region2AMI" : {
"Fn::Include": "https://api.netcubed.de/latest/ami/lookup?architecture=HVM64"
}
},
"Resources": {
"Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"UserData": {
"ImageId": {
"FindInMap": [ "Region2AMI", { "Ref": "AWS::Region" }, "AMI" ]
},
"Fn::Base64": {
"Fn::Sub": {
"Fn::Include": {
"type": "string",
"location": "userdata.sh"
} } } } } } } }
This is what the userdata.sh
looks like:
#!/bin/bash
/opt/aws/bin/cfn-init -s ${AWS::StackId} -r MyInstance --region ${AWS::Region}
cfn-include synopsis.json > output.template
# you can also compile remote files
cfn-include https://raw.githubusercontent.com/monken/cfn-include/master/examples/synopsis.json > output.template
Alternatively, you can compile the template using the web service
curl -Ssf -XPOST https://api.netcubed.de/latest/template -d '{"Fn::Include":"https://raw.githubusercontent.com/monken/cfn-include/master/examples/synopsis.json"}' > output.template
The output will be something like this:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Mappings": {
"Region2AMI": {
"Metadata": {
"Name": "amzn-ami-hvm-2016.09.0.20161028-x86_64-gp2",
"Owner": "amazon",
"CreationDate": "2016-10-29T00:49:47.000Z"
},
"us-east-2": {
"AMI": "ami-58277d3d"
},
// ...
} },
"Resources": {
"Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": {
"FindInMap": [ "Region2AMI", { "Ref": "AWS::Region" }, "AMI" ]
},
"UserData": {
"Fn::Base64": {
"Fn::Sub": {
"Fn::Join": ["", [
"#!/bin/bash\n",
"\"/opt/aws/bin/cfn-init -s ${AWS::StackId} -r MyInstance --region ${AWS::Region}\n",
""
] ] } } } } } } }
Place Fn::Include
anywhere in the template and it will be replaced by the contents it is referring to. The function accepts an object. Parameters are:
- location: The location to the file can be relative or absolute. A relative location is interpreted relative to the template. Included files can in turn include more files, i.e. recursion is supported.
- type (optional): either
json
,string
orapi
. Defaults tojson
.string
will include the file literally which is useful in combination withFn::Sub
.api
will call any AWS API and return the response which can be included in the template. Choosejson
for both JSON and YAML files. Theliteral
type is deprecated and uses the infamousFn::Join
syntax. - context (optional, deprecated): If
type
isliteral
a context object with variables can be provided. The object can contain plain values or references to parameters or resources in the CloudFormation template (e.g.{ "Ref": "StackId" }
). Use Mustache like syntax in the file. This option is deprecated in favor of theFn::Sub
syntax (see examples below). - query (optional): If
type
isjson
a JMESPath query can be provided. The file to include is then queried using the value as a JMESPath expression.
Only applicable if type is api
:
- service: Service to call (see AWSJavaScriptSDK, case sensitive, e.g.
EC2
,CloudFormation
) - action: Action to call (case sensitive, e.g.
updateStack
,describeRegions
) - parameters (optional): Parameters passed to action (e.g.
{ StackName: "MyStack" }
) - region (optional): Either
AWS_DEFAULT_REGION
or this parameter have to be set which specifies the region where the API call is made.
You can also use a plain string if you want the default behavior, which is simply including a JSON file.
Include a file from a URL
{ "Fn::Include": "https://example.com/include.json" }
// equivalent to
{ "Fn::Include": {
"type": "json",
"location": "https://example.com/include.json"
}
}
Include a file from an S3 bucket. Authentication is handled by aws-sdk
. See Setting AWS Credentials for details.
{ "Fn::Include": "s3://bucket-name/include1.json" }
Include a file in the same folder
{ "Fn::Include": "include.json" }
Include a file literally
{ "Fn::Sub": {
"Fn::Include": {
"type": "string",
"location": "https://example.com/userdata.txt"
} } }
Include an AWS API response, e.g. loop through all regions and return the image id of a specific AMI:
{ "Fn::Merge": {
"Fn::Map": [{
"Fn::Include": {
"type": "api",
"service": "EC2",
"action": "describeRegions",
"query": "Regions[*].RegionName[]"
} }, {
"_": {
"AMI": {
"Fn::Include": {
"type": "api",
"service": "EC2",
"action": "describeImages",
"region": "_",
"query": "Images[*].ImageId | [0]",
"parameters": {
"Filters": [{
"Name": "manifest-location",
"Values": ["amazon/amzn-ami-hvm-2016.03.3.x86_64-gp2"],
}]
} } } } } ] } }
{ "ap-south-1": { "AMI": "ami-ffbdd790" },
"eu-west-1": {"AMI": "ami-f9dd458a" },
"ap-southeast-1": { "AMI": "ami-a59b49c6" },
...
}
Fn::Map
is the equivalent of the JavaScript map()
function allowing for the transformation of an input array to an output array.
By default the string _
is used as the variable in the map function. A custom variable can be provided as a second parameter, see Fn::Flatten
for an example. If a custom variable is used, the variable will also be replaced if found in the object key, see Fn::Merge
for an example.
{
"Fn::Map": [
[80, 443],
{
"CidrIp": "0.0.0.0/0",
"FromPort": "_",
"ToPort": "_"
}
]
}
[{
"CidrIp": "0.0.0.0/0",
"FromPort": "80",
"ToPort": "80"
}, {
"CidrIp": "0.0.0.0/0",
"FromPort": "443",
"ToPort": "443"
}]
This function flattens an array a single level. This is useful for flattening out nested Fn::Map
calls.
{
"Fn::Flatten": {
"Fn::Map": [
[80, 443], "$", {
"Fn::Map": [
["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"], {
"CidrIp": "_",
"FromPort": "$",
"ToPort": "$"
}
]
}
]
}
},
Results in:
[{
"CidrIp": "10.0.0.0/8",
"FromPort": "80",
"ToPort": "80"
}, {
"CidrIp": "172.16.0.0/12",
"FromPort": "80",
"ToPort": "80"
}, {
"CidrIp": "192.168.0.0/16",
"FromPort": "80",
"ToPort": "80"
}, {
"CidrIp": "10.0.0.0/8",
"FromPort": "443",
"ToPort": "443"
}, {
"CidrIp": "172.16.0.0/12",
"FromPort": "443",
"ToPort": "443"
}, {
"CidrIp": "192.168.0.0/16",
"FromPort": "443",
"ToPort": "443"
}]
Fn::Merge
will merge an array of objects into a single object. See lodash / merge for details on its behavior.
For example, this allows you to merge objects of your template that have been created with Fn::Map
. This snippet shows how multiple subnets can be created for each AZ and then merged with the rest of the template.
{
"Resources": {
"Fn::Merge": {
"Fn::Flatten": [{
"Fn::Map": [
["A", "B"], "AZ", {
"Subnet${AZ}": {
"Type": "AWS::EC2::Subnet"
}
}
]
}, {
"SG": {
"Type": "AWS::EC2::SecurityGroup"
}
}]
}
}
}
{
"Resources": {
"SubnetA": {
"Type": "AWS::EC2::Subnet"
},
"SubnetB": {
"Type": "AWS::EC2::Subnet"
},
"SG": {
"Type": "AWS::EC2::SecurityGroup"
}
}
}
Fn::Stringify
will take the passed value and transform it to a JSON string. This is useful for parameters that require a JSON document as a string. Using this function, you can keep writing your configuration in YAML and let the function transform it into a JSON string.
Another useful application is the use of this function in a config file passed as --cli-input-json
parameter.
# myconfig.yml
StackName: MyStack
TemplateBody: !Stringify { Fn::Include: mytemplate.yml }
Parameters:
- ParameterKey: Foo
ParameterValue: Bar
You can then simply run the following command to deploy a stack:
cfn-include myconfig.yml | jq 'del(.Metadata)' > myconfig.json
aws cloudformation create-stack --cli-input-json file://myconfig.json
See /examples for templates that call an API Gateway endpoint to collect AMI IDs for all regions. There is also a good amount of tests that might be helpful.
A common pattern is to process a template, validate it against the AWS validate-template API, minimize it and upload the result to S3. You can do this with a single line of code:
cfn-include example.template -t -m | aws s3 cp - s3://bucket-name/output.template
cfn-include
honors proxy settings defined in the https_proxy
environmental variable. The module will attempt to load proxy-agent
. Make sure proxy-agent
is installed since it is not a dependency for this module.
Node.js versions 0.10 and up are supported both on Windows and Linux.