wri/gfw-iac-workshop

Materials needed for exercises associated with a Terraform/Infrastructure as Code workshop for the World Resources Institute (WRI).

gfw-iac-workshop

This repository contains the materials needed for exercises associated with a Terraform/Infrastructure as Code workshop for the World Resources Institute (WRI).

• Overview
• Getting started
  • Dependencies
  • Instructions
• Gotchas

Overview

The terraform directory contains a Terraform project, along with four Terraform modules:

• API Gateway, Fargate, and RDS (api-gateway-fargate)
• API Gateway, Lambda, and S3 (api-gateway-lambda-s3)
• CloudFront, S3, and Lambda@Edge (cloudfront-s3-lambda-at-edge)
• Virtual Private Cloud (vpc)

The src directory contains a small Flask application for use with the api-gateway-fargate Terraform module.

The scripts directory contains three scripts:

• cibuild is responsible for building a Flask container image for the application in src.
• cipublish is responsible for publishing that container image to Amazon ECR.
• infra is a wrapper for the terraform command that also manages initialization.

Getting started

Dependencies

• AWS CLI 1.16+
• Docker 19.03+
• Docker Compose 1.24+

Instructions

First, copy the following file, renaming it to gfw-iac-workshop.tfvars in the process:

cp terraform/gfw-iac-workshop.tfvars.example terraform/gfw-iac-workshop.tfvars

Then, customize its contents with a text editor:

• For project, use your name in title case.
• For bucket_name, use a unique name that is unlikely to collide with other bucket names in S3.
• For aws_key_name, follow these instructions to generate a custom EC2 key pair for yourself. Assign the key pair name you select as the value for aws_key_name.
• For ecr_repository_uri, create a repository with the AWS CLI and use the value for repositoryUri from the output:

export AWS_PROFILE=wri
aws ecr create-repository --repository-name hello-repository
{
    "repository": {
        "repositoryArn": "arn:aws:ecr:us-east-1:942210422222:repository/hello-repository",
        "registryId": "942210422222",
        "repositoryName": "hello-repository",
        "repositoryUri": "942210422222.dkr.ecr.us-east-1.amazonaws.com/hello-repository",
        "createdAt": 1572919870.0,
        "imageTagMutability": "MUTABLE",
        "imageScanningConfiguration": {
            "scanOnPush": false
        }
    }
}

Here's an example of a customized gfw-iac-workshop.tfvars:

project = "Peppa"
environment = "Staging"
bucket_name = "PeppaPig"
aws_key_name = "wri-iac-workshop"
ecr_repository_uri = "942210422222.dkr.ecr.region.amazonaws.com/hello-repository"

Next, build a container image for the Flask application (cibuild), then publish it to Amazon ECR (cipublish):

export GFW_IAC_WORKSHOP_AWS_ECR_ENDPOINT="942210422222.dkr.ecr.us-east-1.amazonaws.com/hello-repository"
./scripts/cibuild
./scripts/cipublish

Lastly, launch an instance of the included Terraform container image:

docker-compose build
docker-compose run --rm terraform
bash-5.0#

Once inside the context of the container image, use infra to generate a Terraform plan:

bash-5.0# ./scripts/infra plan

Gotchas

Lambda@Edge functions are replicated to CloudFront edge nodes to localize execution. This leads to an error when Terrafrom attempts to destroy the Lambda function associated with a distribution. In the short-term, the best workaround appears to be to wait for ~1 hour after a destroy attempt fails, then try again.

See: hashicorp/terraform-provider-aws#1721

gfw-iac-workshop's Issues

Create slides that provide an introduction to Terraform

Create a set of slides that go through:

• Infrastructure as code
  • HCL
• Workflow
• Compare vs. CloudFormation
• Install
• AWS provider
  • Configuration
• Workflow
  • Init
  • Fmt
  • Plan
  • Apply
• State as JSON
• Workflow
  • Tweak
  • Plan
  • Apply
  • Destroy
• Dependencies
• cloud-init
• Variables
• Outputs
• Modules
• Remote state

See: https://learn.hashicorp.com/terraform

VPC Terraform module

Create a Terraform module that encapsulates the resources necessary to support a multi-availability zone VPC. This will include a set of both public and private subnets, a bastion host as the entrypoint for managing resources within the VPC, and NAT gateways to facilitate ingress and egress traffic originating from private subnets.

This should look a lot like https://github.com/azavea/terraform-aws-vpc. Apply scrutiny to cut excess resources as needed.

API Gateway, Fargate, and RDS Terraform module

Create a Terraform module that encapsulates the resources necessary to support a container based web service on Fargate. In addition, the Fargate service is expected to consume configuration that provides it with the endpoint needed to read and write to an RDS for PostgreSQL database within the private network.

This deliverable will include the source code for a small Python based web service and an accompanying Dockerfile. The resulting container image will be published to ECR and referenced within ECS task definitions. Those task definitions will be instantiated as ECS services, which will register with a load lancer.

See: https://aws.amazon.com/blogs/compute/access-private-applications-on-aws-fargate-using-amazon-api-gateway-privatelink/

API Gateway, Lambda, S3 Terraform module

Create a Terraform module that encapsulates the resources necessary to support a Lambda based service that is associated with an API Gateway endpoint that proxies requests to a Lambda function. In addition, the Lambda function will inherit the appropriate IAM credentials to read and write to S3.

This should include the source code for a small Python based application to prove S3 connectivity.

See:

• https://learn.hashicorp.com/terraform/aws/lambda-api-gateway
• https://github.com/azavea/dynamic-tileserver-demos/tree/develop/terraform
• https://github.com/azavea/terraform-tegola-lambda

CloudFront, S3, and Lambda@Edge Terraform module

Create a Terraform module that encapsulates the resources necessary to support an S3 bucket fronted by a CloudFront distribution. This will also include a snippet of JavaScript script source code to register with CloudFront distribution request/response cycle events via Lambda@Edge (maybe just something that tweaks the response headers).

See: https://github.com/azavea/cicero-live/tree/develop/deployment/terraform

Deliver on-site workshop

Create slides for a base software delivery pipeline

Ensure that the following components are covered:

• Development environment
• STRTA
• CI w/ Jenkinsfile example
• Infrastructure as code
• Environment namespacing
• Shared remote state management