This Workshop will guide you step by step on how to create your own CloudFormation Hook that internally uses cfn guard to evaluate the resources being provided.
- initial set up Cloud9
- Initiate your project
- Define your Hook
- Adding Guard Library
- Writing our Cfn Guard Rules
- Contract Testing
- Registering your Hook
First navigate to Cloud9 Console click Create environment
.
Next, let's configure this enviroment:
- name:
guard-hook-dev
- EC2 Instance Type:
m5.large
- Platform:
Ubuntu Server
Leave the rest of the values as they where and click create
to start provisioning the environment.
After a few minutes your environment will be ready to run, click open
to run Cloud9 on a new browser tab.
At this point you have your IDE ready to go. Quick notes:
File tree
Main code area
Terminal
Now, let's inscrease the disk size of our Cloud9 environment. First, let's download the resize script
wget https://raw.githubusercontent.com/brianterry/resource-type-workshop/main/resize.sh
Then let's run it to resize to 50GB
chmod +x resize.sh
./resize.sh 50
Follow instaltion steps according to your environment
On the terminal,
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/aws-cloudformation/cloudformation-guard/main/install-guard.sh | sh
If you try to run cfn-guard
you will get a command not found
(1) error, we need to add an enviroment variable (2) by running the following command:
export PATH=${PATH}:~/.guard/bin
Then if you run the command again, you will be able to see an output (3).
Installation documentation can be found here.
For Cloud9, let's run the following command to install cfn cli
from pip
:
pip3 install cloudformation-cli cloudformation-cli-python-plugin
# other avaiable plugins: cloudformation-cli-java-plugin cloudformation-cli-go-plugin cloudformation-cli-typescript-plugin
Veryfi installation by running cfn --version
, you should see something like this:
The first step in creating your hook project is to initiate your project. You can use the CloudFormation CLI init command to initiate your hook project.
The init command launches a wizard that walks you through setting up the project.
Follow these steps to initiate your hook project:
In the CloudFormation CLI, create a directory inside the CloudFormationCLI/ that stores your hook project. This example uses the directory name guard-hook
.
Create a directory named guard-hook
that will stores your project.
mkdir guard-hook
cd guard-hook
Use the init command to initialize a CloudFormation project.
cfn init
this will return
Initializing new project
- The
init
command launches a wizard that walks you through setting up the project. When prompted, enterh
to specify a hooks project.
Do you want to develop a new resource(r) or a module(m) or a hook(h)?.
>> h
- Enter your hook type identifier. For example, Workshop::TestGuard::Hook
What's the name of your hook type?
(Organization::Service::Hook)
>> Workshop::TestGuard::Hook
- The wizard then enables you to select the appropriate language plugin. Python 3.7 to 3.9 are supported.
Select a language for code generation:
[1] python37
[2] python38
[3] python39
(enter an integer):
>> 3
- Choose docker for platform-independent packaging. While docker isn't required, it's highly recommended to make development easier.
Use docker for platform-independent packaging (Y/n)?
This is highly recommended unless you are experienced
with cross-platform Python packaging.
>> Y
This is an overview of the whole wizard
- List the content of the directory by using the ls command.
# if using bash
ls -la
# if using Powershell
ls
List the content of the directory by using the ls command.
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 8/21/2023 1:36 PM docs
d---- 8/21/2023 1:36 PM src
-a--- 8/21/2023 1:36 PM 1888 .gitignore
-a--- 8/21/2023 1:36 PM 676 .rpdk-config
-a--- 8/21/2023 1:36 PM 1264 hook-role.yaml
-a--- 8/21/2023 1:36 PM 1882 README.md
-a--- 8/21/2023 1:36 PM 38 requirements.txt
-a--- 8/21/2023 1:36 PM 4348 rpdk.log
-a--- 8/21/2023 1:36 PM 650 template.yml
-a--- 8/21/2023 1:36 PM 1147 workshop-testguard-hook.json
In this section, we will model our hook. In this example we will create hook for AWS CloudFormation that describes Security Group.
When you initiate the hook project, an example hook schema file is included to help start you modeling your hook. In the case of our example hook, the schema file is named workshop-testguard-hook.json
.
Open this file in your code editor, it shoudl look similar to this one:
typeConfigurations
are extra properties or parameters we can pass to owr hook, in Our case it could be theVpcId
so all the security groups are created only in that one VPC.handlers
, here you can configure where in the life cycle you and to what resources you want you hook to be applied.
sample default configuration file
{
"typeName": "Workshop::TestGuard::Hook",
"description": "Example resource SSE (Server Side Encryption) verification hook",
"sourceUrl": "https://github.com/aws-cloudformation/example-sse-hook",
"documentationUrl": "https://github.com/aws-cloudformation/example-sse-hook/blob/master/README.md",
"typeConfiguration": {
"properties": {
"EncryptionAlgorithm": {
"description": "Encryption algorithm for SSE",
"default": "AES256",
"type": "string"
}
},
"additionalProperties": false
},
"required": [],
"handlers": {
"preCreate": {
"targetNames": [
"AWS::S3::Bucket"
],
"permissions": []
},
"preUpdate": {
"targetNames": [
"AWS::S3::Bucket"
],
"permissions": []
},
"preDelete": {
"targetNames": [
"AWS::S3::Bucket"
],
"permissions": []
}
},
"additionalProperties": false
}
There are a couple of changes we need to do, one of them is to change all the instances of AWS::S3::Bucket
for AWS::EC2::SecurityGroup
.
This will let CloudFormation knows that we want to excecute this hook when Security Groups are present.
Last, remove the preDelete
handler.
Now let's work on typeConfiguration.
First, replace the content of typeConfiguration
for the following snippet:
{
"properties": {
"VpcId": {
"description": "VPC where Security groups can be created",
"default": "authorized-vpc",
"type": "string"
}
},
"additionalProperties": false
}
The file should look like this:
And replace all the instances of AWS::S3::Bucket
with AWS::EC2::SecurityGroup
updated config file
{
"typeName": "Workshop::TestGuard::Hook",
"description": "Example resource SSE (Server Side Encryption) verification hook",
"sourceUrl": "https://github.com/aws-cloudformation/example-sse-hook",
"documentationUrl": "https://github.com/aws-cloudformation/example-sse-hook/blob/master/README.md",
"typeConfiguration": {
"properties": {
"VpcId": {
"description": "VPC where Security groups can be created",
"default": "defaultVPC",
"type": "string"
}
},
"additionalProperties": false
},
"required": [],
"handlers": {
"preCreate": {
"targetNames": [
"AWS::EC2::SecurityGroup"
],
"permissions": []
},
"preUpdate": {
"targetNames": [
"AWS::EC2::SecurityGroup"
],
"permissions": []
}
},
"additionalProperties": false
}
Finally, run cfn generate
to update hook configuration files
cfn generate
This command should output:
Generated files for MyCompany::Testing::hook
This command will auto generate source code and configuration files based on your workshop-testguard-hook.json
file.
For our Hook to easily consume guard rules, we are going to add a couple of libraries like cfn_guard_rs_hook
On our guard-hook/requirements.txt
file, lets add Jinga2
, cfn_guard_rs
and cfn_guard_rs_hook
.
Our requirements.txt
should look like this:
cloudformation-cli-python-lib>=2.1.9
Jinja2==3.1.2
cfn_guard_rs>=0.1.0
cfn_guard_rs_hook>=0.1.0
Then, add the required libraries by running the following command:
pip3 install -r requirements.txt
Then, let's locate the file ./src/workshop_testguard_hook/handlers.py
. This file is the autogenerated logic for a python hook. This is the type of hook where you write your rules as python code. In our case, we will using Guard rules to define what to evaluate.
Delete the content of the file, and lets write the new version.
# Importing logging library
import logging
# Importing guard hook library, this one evaluate cfn-guard rules under the hood
from cfn_guard_rs_hook import GuardHook
# We will create this folder latter in this lab,
# here we will include all the cfn-guard rules
# we want to use for this hook's resources
from . import rules as Rules
# We will create this configuration later on this lab
from .models import TypeConfigurationModel
# Use this logger to forward log messages to CloudWatch Logs.
LOG = logging.getLogger(__name__)
# your hook's type name, it can be found on .rpdk-config file
TYPE_NAME = "Workshop::TestGuard::Hook"
hook = GuardHook(TYPE_NAME, TypeConfigurationModel, Rules)
test_entrypoint = hook.test_entrypoint
NOTE: on the handlers.py
line 10 we can se a warning from the IDE, don't worry, we are going to fix it soon.
Finally, we want to build the project by running the submit command
cfn submit --dry-run
In order to evaluate the rules we will be creating, we need a place to store them and a way for python to know where to find them.
note: the follwing actions can be done using the ui.
Create a rules
file in the ./src/workshop_testguard_hook
mkdir ./src/workshop_testguard_hook/rules
In the rules folder create an empty __init__.py
file. This will allow python to know this folder can be imported.
touch ./src/workshop_testguard_hook/rules/__init__.py
Let's create our fist rule file security-group.rules
on the newly created rules
folder.
./src/workshop_testguard_hook/rules/security-group.rules
touch src/workshop_testguard_hook/rules/security-group.rules
In order to write Guard rules, we need to learn some basics CloudFormation Guard.
AWS CloudFormation Guard is an open-source general-purpose policy-as-code evaluation tool. It provides developers with a simple-to-use, yet powerful and expressive domain-specific language (DSL) to define policies and enables developers to validate JSON- or YAML- formatted structured data with those policies.
let sgs = Resources.*[ Type == 'AWS::EC2::SecurityGroup' ]
rule private_security_group_ingress when %sgs !empty {
when %sgs.Properties.SecurityGroupIngress !empty {
%sgs.Properties {
SecurityGroupIngress[*] {
CidrIp empty
<<
SecurityGroup Ingress must be from another security group,
submit a ticket on support.company.com if you need an internet facing security group
>>
CidrIpv6 empty
<<
SecurityGroup Ingress must be from another security group,
call your admin if your need an internet facing security group
>>
}
}
}
}
rule private_security_group_egress when %sgs !empty {
when %sgs.Properties.SecurityGroupEgress !empty {
%sgs.Properties {
SecurityGroupEgress[*] {
CidrIp empty
<<
SecurityGroup Egrees must be to another security group,
call your admin if your need an internet facing security group
>>
CidrIpv6 empty
<<
SecurityGroup Egress must be tp another security group,
call your admin if your need an internet facing security group
>>
}
}
}
}
The cfn_guard_rs_hook
library uses Jinja to configure Guard rules, so we can use the values from typeConfiguration
properties on the cfn guard template.
Given the following typeConfiguration
:
{
"typeConfiguration": {
"properties": {
"VpcId": {
"description": "VPC where Security groups can be created",
"default": "defaultVPC",
"type": "string"
}
},
"additionalProperties": false
},
}
Now that you've coded your handler functions, it's time to test your hook with contract test. As you model and develop your hook, you should have the CloudFormation CLI perform tests to ensure that the hook is behaving as expected during each event.
Each handler and target are tested twice. One force SUCCESS
once for FAILED
For SUCCESS
response case:
- Status must be SUCCESS.
- Must not return an error code.
- Callback delay should be set to 0 seconds, if specified.
For FAILED
response case:
- Status must be FAILED.
- Must return an error code.
- Must have a message in response.
- Callback delay should be set to 0 seconds, if specified.
For IN_PROGRESS
response case:
- Must not return an error code.
- Result field must not be set in response.
There are 2 ways of specifying input data, one is input files
the second is overrides file
.
more detail on Specifying input data for use in contract tests page
Before creating the Override files
, let's create an inputs
folder on ./guard-hook
.
mkdir inputs
touch inputs/inputs_1_pre_create.json
touch inputs/inputs_1_pre_update.json
touch inputs/inputs_n_pre_delete.json
touch inputs/inputs_1_invalid.json
Here are the content we need to add to each file:
{
"AWS::EC2::SecurityGroup": {
"resourceProperties": {
"GroupDescription": "valid SG",
"GroupName": "validSG",
"SecurityGroupEgress": [
{
"SourceSecurityGroupId": "sg-validId1",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"SourceSecurityGroupId": "sg-validId2",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"VpcId": "vpc-authotizedvpc"
}
}
}
{
"AWS::EC2::SecurityGroup": {
"resourceProperties": {
"GroupDescription": "valid SG",
"GroupName": "validSG",
"SecurityGroupEgress": [
{
"SourceSecurityGroupId": "sg-validId3",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"SourceSecurityGroupId": "sg-validId4",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"VpcId": "vpc-authotizedvpc"
},
"previousResourceProperties": {
"GroupDescription": "valid SG",
"GroupName": "validSG",
"SecurityGroupEgress": [
{
"SourceSecurityGroupId": "sg-validId1",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"SourceSecurityGroupId": "sg-validId2",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"VpcId": "vpc-authotizedvpc"
}
}
}
{
"AWS::EC2::SecurityGroup": {
"resourceProperties": {
"GroupDescription": "valid SG",
"GroupName": "validSG",
"SecurityGroupEgress": [
{
"SourceSecurityGroupId": "sg-validId1",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"SourceSecurityGroupId": "sg-validId2",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"VpcId": "vpc-authotizedvpc"
}
}
}
{
"AWS::EC2::SecurityGroup": {
"resourceProperties": {
"GroupDescription": "invalid SG",
"GroupName": "invalidSG",
"SecurityGroupEgress": [
{
"CidrIp": "192.168.1.122",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"CidrIp": "192.168.1.123",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"VpcId": "vpc-unauthotizedvpc"
}
}
}
To test your hook locally, you will need to create a typeConfiguration.json
and save it in ~/.cfn-cli/
typeConfiguration.json In this file we can specify 3 key elements.
- TargetStack: either
NONE
orALL
, this is like theon
off
switch for your hook. - FailureMode: either
WARN
orFAIL
- Properties: properties defined when modeling the hook
Let's use this file as configuration, here we are telling CloudFormation to fail
if the resource is not compliant, and apply this hook to all
stack.
first create a local typeConfiguration.json
file:
touch typeConfigurations.json
Use the following configuration as an starting point:
{
"CloudFormationConfiguration": {
"HookConfiguration": {
"TargetStacks": "ALL",
"FailureMode": "FAIL",
"Properties": {
"VpcId": "validVPCID"
}
}
}
}
more detail here
Open a second terminal Alt + t
(on Cloud9), and activate the python environment
then cd into guard-hook
folder and start the sam lambda execution:
cd guard-hook
sam local start-lambda
now in the first terminal let's execute the test by running:
cfn test -v --typeconfig typeConfigurations.json
note: error on pre_delete hook is a know but, it won't stop the flow of the lab
Register your hook by using the submit command.
cfn submit --set-default
You can view you registered hooks by running the following command:
aws cloudformation list-types
Locate the current hook in the result, look for "TypeName": "Workshop::TestGuard::Hook"
and take note of the TypeArn
, we will use it in the next step.
Now, we need to get the current VPC id, we can get it from the terminal by running:
aws ec2 describe-vpcs
Let's activate our hook using aws cli.
replace your __YOUR_TYPE_ARN__
with its corresponding value.
aws cloudformation --region us-east-1 set-type-configuration --configuration '{"CloudFormationConfiguration":{"HookConfiguration":{"TargetStacks":"ALL","FailureMode":"FAIL","Properties":{ }}}}' --type-arn __YOUR_TYPE_ARN__
After this command, you should see a similar output:
{
"ConfigurationArn": "arn:aws:cloudformation:us-east-2:__account_id__:type-configuration/hook/Workshop-TestGuard-Hook/default"
}
Finally, we can view our Hook on the CloudFormation Registry console you will be able to see the registered hook.
We are going to create the main Security Group, this one will be used as source and target of our test. Imagine this SG is given to you by other organization and you want to create your own security groups that can talk with this one.
First, let's open the EC2 Security Groups AWS Console and click on Create Security Group
.
Give it a name, it can be guard-hook-main-sg
. Add a description and click on Create security group
.
Then copy the Security Group ID
and Vpc ID
, we will use them on the next step.
In order to see our guard-hook
in action, we neet to deploy a compliant
and non-compliant
stacks. Let's first create a templates
folder on ./guard-hook
.
Then lets create the following files inside templates
, replace the DestinationSecurityGroupId
and SourceSecurityGroupId
for the id you copied on the previous step :
mkdir templates
touch templates/compliant.json
touch templates/non-compliant.json
compliant.json
{
"Parameters": {},
"Resources": {
"SG": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "valid SG",
"GroupName": "validSG",
"SecurityGroupEgress": [
{
"DestinationSecurityGroupId": "__copied from previous step__",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"SourceSecurityGroupId": "__copied from previous step__",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"VpcId": "__copied from previous step__"
}
}
}
}
non-compliant.json
{
"Parameters": {},
"Resources": {
"SG": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "invalid SG",
"GroupName": "invalidSG",
"SecurityGroupEgress": [
{
"CidrIp": "192.168.1.122",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"CidrIp": "192.168.1.123",
"FromPort": 443,
"ToPort": 3000,
"IpProtocol": "tcp"
}
],
"VpcId": "__copied from previous step__"
}
}
}
}
Now that we have both templates ready, lets start by publishing the compliant
stack.
aws cloudformation create-stack --stack-name compliant --template-body file://templates/compliant.json
You should get a similar output:
{
"StackId": "arn:aws:cloudformation:us-east-2:__account_id__:stack/compliant/0bb44e80-4127-11ee-9b03-0282e8f12ae7"
}
If you check your CloudFormation console you should see the stack being created
Now let's test the non compliant
stack:
aws cloudformation create-stack --stack-name non-compliant --template-body file://templates/non-compliant.json
You should get a similar output:
{
"StackId": "arn:aws:cloudformation:us-east-2:__account_id__:stack/non-compliant/7fc91db0-412b-11ee-bfa9-0362fddb0cd3"
}
Now, on the CloudFormation console we can see this template failed.
Congratulations! you have created, published and tested your new guard-hook
!
Now you can start experimenting with more complex guard rules. Check this link for more information.
Also check how to deploy hooks to multiple accounts using Stack Sets.
Thank you!