AWS Cloudformation templates are an incredibly powerful way to build sets of resources in Amazon's AWS environment. Unfortunately, because they are specified in JSON or YAML, they are also difficult to write and maintain:
-
JSON does not allow comments, although CloudFormation now supports YAML
-
JSON/YAML templates do not allow re-usable logic
-
It is easy for a person reading a template to miss the association and dependencies between entries
-
References and internal functions have a particularly unpleasant syntax.
The cnfdsl gem provides a DSL that allows you to write templates using friendly Ruby logic to generate and validate cloudformation templates.
ruby version > 2.7 is required to run cfndsl, you should look at using rbenv example for installing with rbenv
rbenv exec gem install cfndsl
Example for doing it system wide Ruby
sudo gem install cfndsl
Update the the cloudformation specification to the latest version.
cfndsl -u
or update to a specific version
cfndsl -u 7.1.0
Now write a template in the dsl
CloudFormation {
Description "Test"
Parameter("One") {
String
Default "Test"
MaxLength 15
}
Output(:One,FnBase64( Ref("One")))
EC2_Instance(:MyInstance) {
ImageId "ami-12345678"
}
}Then run cfndsl on the file
chris@raspberrypi:~/git/cfndsl$ cfndsl test.rb | json_pp
{
"Parameters" : {
"One" : {
"Type" : "String",
"Default" : "Test",
"MaxLength" : 15
}
},
"Resources" : {
"MyInstance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-12345678"
}
}
},
"AWSTemplateFormatVersion" : "2010-09-09",
"Outputs" : {
"One" : {
"Value" : {
"Fn::Base64" : {
"Ref" : "One"
}
}
}
},
"Description" : "Test"
}
Aside: that is correct - a significant amount of the development for this gem was done on a Raspberry Pi.
cfndsl comes with a number of helper methods defined on each resource and/or the stack as a whole.
Metadata is a special template section described here. The argument supplied must be JSON-able. Some CloudFormation features reference special keys if included in the Metadata, check the AWS documentation for specifics.
CloudFormation do
Metadata(foo: 'bar')
EC2_Instance(:myInstance) do
ImageId 'ami-12345678'
InstanceType 't1.micro'
end
endAt a bare minumum, parameters need a name, and default to having Type String. Specify the parameter in the singular, not plural:
CloudFormation do
Parameter 'foo'
end{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"foo": {
"Type": "String"
}
}
}However, they can accept all of the following additional keys per the documentation:
Parameter('foo') do
Description 'This is a sample parameter definition'
Type 'String'
Default 'foo'
NoEcho true
AllowedValues %w(foo bar)
AllowedPattern '/pattern/'
MaxLength 5
MinLength 3
MaxValue 10
MinValue 2
ConstraintDescription 'The error message printed when a parameter outside the constraints is given'
endParameters can be referenced later in your template:
EC2_Instance(:myInstance) do
InstanceType 'm3.xlarge'
UserData Ref('foo')
endMappings are a hash-based lookup for your template. They can be specified in the singular or plural.
CloudFormation do
Mapping('foo', letters: { a: 'a', b: 'b' }, numbers: { 1: 1, 2: 2 })
end{
"AWSTemplateFormatVersion": "2010-09-09",
"Mappings": {
"foo": {
"letters": {
"a": "a",
"b": "b"
},
"numbers": {
"one": 1,
"two": 2
}
}
}
}
}You can then reference them later in your template using the FnFindInMap method:
EC2_Instance(:myInstance) do
InstanceType 'm3.xlarge'
UserData FnFindInMap('foo', :numbers, :one)
endOutputs are declared one at a time and must be given a name and a value at a minimum, description is optional. Values are most typically obtained from other resources using Ref or FnGetAtt:
CloudFormation do
EC2_Instance(:myInstance) do
ImageId 'ami-12345678'
Type 't1.micro'
end
Output(:myInstanceId) do
Description 'My instance Id'
Value Ref(:myInstance)
end
end{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"myInstance": {
"Properties": {
"ImageId": "ami-12345678"
},
"Type": "AWS::EC2::Instance"
}
},
"Outputs": {
"myInstanceId": {
"Description": "My instance Id",
"Value": {
"Ref": "myInstance"
}
}
}
}Conditions must be created with statements in three sections: a variable entry as a Parameter, a template-level Condition that holds the logic based upon the value of that Parameter, and a resource-level Condition that references the template-level one by logical id.
CloudFormation do
Parameter(:environment) do
Default 'development'
AllowedValues %w(production development)
end
Condition(:createResource, FnEquals(Ref(:environment), 'production'))
EC2_Instance(:myInstance) do
Condition :createResource
ImageId 'ami-12345678'
Type 't1.micro'
end
end{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"environment": {
"Type": "String",
"Default": "development",
"AllowedValues": [
"production",
"development"
]
}
},
"Conditions": {
"createResource": {
"Fn::Equals": [
{
"Ref": "environment"
},
"production"
]
}
},
"Resources": {
"myInstance": {
"Condition": "createResource",
"Properties": {
"ImageId": "ami-12345678"
},
"Type": "AWS::EC2::Instance"
}
}
}Cfndsl creates accessor methods for all of the resources listed here and here. If a resource is missing, or if you prefer to explicitly enter a resource in a template, you can do so. Keep in mind that since you are using the generic Resource class, you will also need to explicitly set the Type and that you no longer have access to the helper methods defined on that particular class, so you will have to use the Property method to set them.
CloudFormation do
Resource(:myInstance) do
Type 'AWS::EC2::Instance'
Property('ImageId', 'ami-12345678')
Property('Type', 't1.micro')
end
# Will generate the same json as this
#
# EC2_Instance(:myInstance) do
# ImageId 'ami-12345678'
# Type 't1.micro'
# end
end{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"myInstance": {
"Type": "AWS::ApiGateway::Resource",
"Properties": {
"ImageId": "ami-12345678",
"Type": "t1.micro"
}
}
}
}When using the generic Resource method, rather than the dsl methods, specify the type of resource using Type and the properties using Property. See Template Resources for an example.
Resource conditions are specified singularly, referencing a template-level condition by logical id. See Template Conditions for an example.
Resources can depend upon other resources explicitly using DependsOn. It accepts one or more logical ids.
CloudFormation do
EC2_Instance(:database) do
ImageId 'ami-12345678'
Type 't1.micro'
end
EC2_Instance(:webserver) do
DependsOn :database
ImageId 'ami-12345678'
Type 't1.micro'
end
end{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"database": {
"Properties": {
"ImageId": "ami-12345678"
},
"Type": "AWS::EC2::Instance"
},
"webserver": {
"Properties": {
"ImageId": "ami-12345678"
},
"Type": "AWS::EC2::Instance",
"DependsOn": "database"
}
}
}Resources can have deletion policies associated with them. Specify them one per resource as an attribute:
CloudFormation do
EC2_Instance(:myInstance) do
DeletionPolicy 'Retain'
ImageId 'ami-12345678'
Type 't1.micro'
end
endYou can attach arbitrary metadata as an attribute. Arguments provided must be able to be JSON-ified:
CloudFormation do
EC2_Instance(:myInstance) do
Metadata(foo: 'bar')
ImageId 'ami-12345678'
Type 't1.micro'
end
endThese attributes are only usable on particular resources. The name of the attribute is not arbitrary, it must match the policy name you are trying to attach. Different policies have different parameters.
CloudFormation do
EC2_Instance(:myInstance) do
ImageId 'ami-12345678'
Type 't1.micro'
CreationPolicy(:ResourceSignal, { Count: 1, Timeout: 'PT1M' })
end
endThere is a more detailed example in the samples directory. The file "autoscale.template" is one of the standard Amazon sample templates. "autoscale.rb" generates an equivalent template file.
There's also a larger set of examples available at cfndsl_examples thanks to @neillturner.
The cfndsl command line program now accepts some command line options.
Usage: cfndsl [options] FILE
-o, --output FILE Write output to file
-y, --yaml FILE Import yaml file as local variables
-j, --json FILE Import json file as local variables
-p, --pretty Pretty-format output JSON
-f, --format FORMAT Specify the output format (JSON default)
-D, --define "VARIABLE=VALUE" Directly set local VARIABLE as VALUE
-v, --verbose Turn on verbose output
-m, --disable-deep-merge Disable deep merging of yaml
-s, --specification-file FILE Location of Cloudformation Resource Specification file
-u [VERSION], Update the Resource Specification file to latest, or specific version
--update-specification
-g RESOURCE_TYPE,RESOURCE_LOGICAL_NAME,
--generate Add resource type and logical name
-l, --list List supported resources
-h, --help Display this screen
By default, cfndsl will attempt to evaluate FILE as cfndsl template and print the resulting cloudformation json template to stdout. With the -o option, you can instead have it write the resulting json template to a given file. The -v option prints out additional information (to stderr) about what is happening in the model generation process.
The -y, -j, -r and -D options can be used to control some things about the environment that the template code gets evaluate in. For instance, the -D option allows you to set a variable at the command line that can then be referred to within the template itself.
This is best illustrated with a example. Consider the following cfndsl template
# cfndsl template sample/t1.rb
CloudFormation do
description = external_parameters.fetch(:description, 'default description')
machines = external_parameters.fetch(:machines, 1).to_i
Description description
(1..machines).each do |i|
name = "machine#{i}"
EC2_Instance(name) do
ImageId 'ami-12345678'
Type 't1.micro'
end
end
endNote the two variables description and machines. The template
sets these to some reasonable default values, and if you run cfndsl
on it without changing them in any way you get the following cloudformation
template:
{
"Resources": {
"machine1": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-12345678"
}
}
},
"Description": "default description",
"AWSTemplateFormatVersion": "2010-09-09"
}However if you run the command
$ cfndsl sample/t1.rb -D 'description=3 machine cluster' -D 'machines=3'you get the following generated template.
{
"Resources": {
"machine3": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-12345678"
}
},
"machine2": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-12345678"
}
},
"machine1": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-12345678"
}
}
},
"Description": "3 machine cluster",
"AWSTemplateFormatVersion": "2010-09-09"
}The -y and -j options allow you to group several variable definitions into a single file (formated as either yaml or json respectively). If you had a file called 't1.yaml' that contained the following,
# sample/t1.yaml
description: 5 machine cluster
machines: 5the command
$ cfndsl sample/t1.rb -y sample/t1.yamlwould generate a template with 5 instances declared.
Specifying multiple -y options will default deep_merge all the yaml in the order specified. You can disable this with -m.
Simply add the following to your Rakefile:
require 'cfndsl/rake_task'
namespace(:cfndsl) do
CfnDsl::RakeTask.new do |t|
# Use a custom specification file
t.specification(file: 'tmp/cloudformation_resources.json')
desc 'Generate CloudFormation Json'
t.json(name: :json, files: FileList.new('sample/*.rb'), pathmap: 'tmp/%f.json')
# Generate yaml output, loading an extra file that matches the source file
t.yaml(name: :yaml, files: 'sample/t1.rb', pathmap: 'tmp/%f.yaml', extras: '%X.yaml')
end
endAnd then use rake to generate the cloudformation:
$ bin/rake cfndsl:generate# Optionally before requiring 'cfndsl' set global options
require `cfndsl/globals`
CfnDsl.specification_file(file) # Override location of json spec file
CfnDsl.disable_deep_merge # Prevent monkey patching of Hash with deep merge capabilities
require `cfndsl`
# As a function that takes a block of DSL
template = CloudFormation do
# Some CfnDsl
end
puts JSON.pretty_generate(template.validate)
# As a class that is a template
class MyTemplate < CfnDsl::CloudFormationTemplate
def initialize
super do
# Base DSL
end
end
def instance(logical_id)
this = self
EC2_Instance(logical_id) do
self.class.name # << 'CfnDsl::ResourceDefinition' not 'MyTemplate' !
InstanceType this.instance_type
end
end
#do not make this private!
def instance_type
't3.large'
end
end
# As a builder class
class Builder
include CfnDsl::CloudFormation
def model
this = self # the DSL blocks are executed via instance_eval
ref_param = Ref('param') # and other Fn::*
template = CloudFormation do
# Some DSL
end
template.validate.to_json
end
endBy supplying the -g parameter you are now able to generate cloudformation resources for supported objects, for a list of supported resources run cfndsl -l
Example
cfndsl -g AWS::EC2::EIP,EIPrequire 'cfndsl'
CloudFormation do
Description 'auto generated cloudformation cfndsl template'
EC2_EIP('EIP') do
Domain String # http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-eip.html#cfn-ec2-eip-domain
InstanceId String # http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-eip.html#cfn-ec2-eip-instanceid
end
endMany thanks to the base code from cfnlego to make this possible!
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent