nzoschke/gofaas

A boilerplate Go and AWS Lambda app. Demonstrates an expert configuration of 10+ AWS services to support running Go functions-as-a-service (FaaS).

Go Functions-as-a-Service

Running a Go application on AWS Lambda is easier than ever, once you figure out how to configure Lambda, API Gateway and 10 or other "serverless" services to support the Go functions.

This is a boilerplate app with all the AWS pieces configured correctly and explained in depth. See the docs folder for detailed guides about functions, tracing, security, automation and more with AWS and Go.

With this foundation you can skip over all the undifferentiated setup, and focus entirely on your Go code.

Motivation

Functions-as-a-Service (FaaS) like AWS Lambda are one of the latest advances in cloud Infrastructure-as-a-Service (IaaS). Go is particularly well-suited to run in Lambda due to its speed, size and cross-compiler. Check out the Intro to Go Functions-as-a-Service and Lambda doc for more explanation.

For a long time, Go in Lambda was only possible through hacks -- execution shims, 3rd party frameworks and middleware, and little dev/prod parity. But in January 2018, AWS launched official Go support for Lambda and Go released v1.10 paving the clearest path yet for us Gophers.

This project demonstrates a simple and clean foundation for Go in Lambda. You can clone and deploy it with a few commands to get a feel for the stack. Or you can fork and rework it to turn it into your own web app.

It demonstrates:

Component	Via	Config, Code
HTTP functions	Lambda, API Gateway	💾
Worker functions (one-off and periodic)	Lambda, Invoke API, CloudWatch Events	💾
Development, packaging and deployment	make, go, aws-sam-cli, CloudFormation	⚙️
Per-function environment and policies	Lambda, IAM	⚙️
Custom domains	CloudFront, ACM	⚙️
Static web content	S3, CloudFront, ACM	⚙️
Static web security with Google OAuth 2.0	CloudFront, Lambda@Edge, SSM Parameters	💾
Function security with CORS and JWT	API Gateway, jwt-go	💾
Function traces and logs	CloudWatch Logs, X-Ray, AWS SDKs for Go	💾
Notifications	SNS	💾
Databases and encryption at rest	DynamoDB, KMS	💾
Testing with mock AWS clients	Go interfaces, aws-sdk-go	💾

What's remarkable is how little work is required to get all functionality for our app. We don't need a framework, platform-as-a-service, or even any 3rd party software-as-a-service. And no, we don't need servers. By standing on the shoulders of Go and AWS, all the undifferentiated heavy lifting is managed for us.

We just need an expert CloudFormation config file and a simple Makefile, then we can focus entirely on writing Go functions.

Quick Start

This project uses :

• AWS CLI
• AWS SAM CLI
• Docker CE
• Go 1.10
• watchexec

Install the CLI tools and Docker CE

$ brew install awscli go node python@2 watchexec
$ pip2 install aws-sam-cli
$ open https://store.docker.com/search?type=edition&offering=community

We may want to upgrade existing tools...

 

$ brew upgrade awscli go node python@2 watchexec
$ pip2 install --upgrade aws-sam-cli

We may want to double check the installed versions...

 

$ aws --version
aws-cli/1.16.20 Python/3.7.0 Darwin/17.7.0 botocore/1.12.10

$ sam --version
SAM CLI, version 0.6.0

$ docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:21:31 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:29:02 2018
  OS/Arch:          linux/amd64
  Experimental:     true


$ go version
go version go1.11.1 darwin/amd64

$ watchexec --version
watchexec 1.9.2

We may also want to configure the AWS CLI with IAM keys to develop and deploy our application...

 

Follow the Creating an IAM User in Your AWS Account doc to create a IAM user with programmatic access. Call the user gofaas-admin and attach the "Administrator Access" policy for now.

Then configure the CLI. Here we are creating a new profile that we can switch to with export AWS_PROFILE=gofaas. This will help us isolate our experiments from other AWS work.

Configure an AWS profile with keys and switch to the profile:

$ aws configure --profile gofaas
AWS Access Key ID [None]: AKIA................
AWS Secret Access Key [None]: PQN4CWZXXbJEgnrom2fP0Z+z................
Default region name [None]: us-east-1
Default output format [None]: json

$ export AWS_PROFILE=gofaas
$ aws iam get-user
{
    "User": {
        "Path": "/",
        "UserName": "gofaas-admin",
        "UserId": "AIDAJA44LJEOECDPZ3S5U",
        "Arn": "arn:aws:iam::572007530218:user/gofaas-admin",
        "CreateDate": "2018-02-16T16:17:24Z"
    }
}

Get the App

We start by getting and testing the github.com/nzoschke/gofaas.

$ git clone https://github.com/nzoschke/gofaas.git ~/dev/gofaas
$ cd ~/dev/gofaas

$ make test
go test -v ./...
go: finding github.com/aws/aws-xray-sdk-go v1.0.0-rc.8
go: finding github.com/aws/aws-lambda-go v1.6.0
go: finding github.com/aws/aws-sdk-go v1.15.49
...
=== RUN   TestUserCreate
--- PASS: TestUserCreate (0.00s)
...
ok     github.com/nzoschke/gofaas      0.014s
PASS

This gives us confidence in our Go environment.

Develop the App

We can then build the app and start a development server:

$ make dev
cd ./handlers/dashboard && GOOS=linux go build...
2018/02/25 08:03:12 Connected to Docker 1.35
2018/02/16 07:40:32 Fetching lambci/lambda:go1.x image for go1.x runtime...

Mounting handler (go1.x) at http://127.0.0.1:3000/users/{id} [DELETE]
Mounting handler (go1.x) at http://127.0.0.1:3000/users/{id} [PUT]
Mounting handler (go1.x) at http://127.0.0.1:3000/users/{id} [GET]
Mounting handler (go1.x) at http://127.0.0.1:3000/ [GET]
Mounting handler (go1.x) at http://127.0.0.1:3000/users [POST]

Now we can access our HTTP functions on port 3000:

$ curl http://localhost:3000
<html><body><h1>gofaas dashboard</h1></body></html>

We can also invoke a function directly:

$ echo '{}' | sam local invoke WorkerFunction
...
START RequestId: 36d6d40e-0d4b-168c-63d5-76b25f543d21 Version: $LATEST
2018/02/25 16:05:21 Worker Event: {SourceIP: TimeEnd:0001-01-01 00:00:00 +0000 UTC TimeStart:0001-01-01 00:00:00 +0000 UTC}
END RequestId: 36d6d40e-0d4b-168c-63d5-76b25f543d21
REPORT RequestId: 36d6d40e-0d4b-168c-63d5-76b25f543d21  Duration: 681.67 ms  Billed Duration: 700 ms  Memory Size: 128 MB  Max Memory Used: 14 MB

Note: if you see No AWS credentials found. Missing credentials may lead to slow startup..., review aws configure list and your AWS_PROFILE env var.

This gives us confidence in our development environment.

Deploy the App

Now we can package and deploy the app:

$ make deploy
make_bucket: pkgs-572007530218-us-east-1
Uploading to 59d2ea5b6bdf38fcbcf62236f4c26f21  3018471 / 3018471.0  (100.00%)
Waiting for changeset to be created
Waiting for stack create/update to complete
Successfully created/updated stack - gofaas

ApiUrl	https://x19vpdk568.execute-api.us-east-1.amazonaws.com/Prod

Now we can access our HTTP functions on AWS:

$ curl https://x19vpdk568.execute-api.us-east-1.amazonaws.com/Prod
<html><body><h1>gofaas dashboard</h1></body></html>

We can also invoke a function directly:

$ aws lambda invoke --function-name gofaas-WorkerFunction --log-type Tail --output text --query 'LogResult' out.log | base64 -D
START RequestId: 0bb47628-1718-11e8-ad73-c58e72b8826c Version: $LATEST
2018/02/21 15:01:07 Worker Event: {SourceIP: TimeEnd:0001-01-01 00:00:00 +0000 UTC TimeStart:0001-01-01 00:00:00 +0000 UTC}
END RequestId: 0bb47628-1718-11e8-ad73-c58e72b8826c
REPORT RequestId: 0bb47628-1718-11e8-ad73-c58e72b8826c  Duration: 11.11 ms  Billed Duration: 100 ms  Memory Size: 128 MB  Max Memory Used: 41 MB

Look at that speedy 11 ms duration! Go is faster than the minimum billing duration of 100 ms.

This gives us confidence in our production environment.

Development Environment

If we want to work on the worker or database functions locally, we need to give the functions environment variables with pointers to DynamoDB, KMS and S3. Open up env.json and set BUCKET, etc. with the ids of the resources we just created on deploy:

$ aws cloudformation describe-stack-resources --output text --stack-name gofaas \
  --query 'StackResources[*].{Name:LogicalResourceId,Id:PhysicalResourceId,Type:ResourceType}' | \
  grep 'Bucket\|Key\|UsersTable'

gofaas-bucket-aykdokk6aek8            Bucket      AWS::S3::Bucket
8eb8e209-51fb-41fa-adfe-1ec401667df4  Key         AWS::KMS::Key
gofaas-UsersTable-1CYAQH3HHHRGW       UsersTable  AWS::DynamoDB::Table

Integration Testing

We can verify the app functionality by creating an isolated testing stack, testing all the endpoints, then deleting the stack. The ci.sh script automates this:

$ ./ci.sh
aws cloudformation package ...
aws cloudformation deploy ...
...
<title>My first gofaas/Vue app</title>
"username": "test"
{"ExecutedVersion":null,"FunctionError":null,"LogResult":null,"Payload":"","StatusCode":202}
...
✅ SUCCESS!

Docs

Check out the docs folder where each component is explained in more detail.

Contributing

Find a bug or see a way to improve the project? Open an issue.

License

Apache 2.0 © 2018 Noah Zoschke

gofaas's Issues

More App Functionality

There's still more functionality to port over:

• HTTP Auth and Tokens
• KMS env encryption
• Worker API
• Users API + Table
• KMS data encryption

I'm also considering including:

• S3 / CloudFront static site

This could be a nice demo of multiple stacks and export, where the S3 bucket is an export the app can import.

SQS

SQS is an event source worth demoing and documenting

Lambda challenges / advanced topics

It might be interesting to dive into some more advanced topics and challenges with building serverless apps:

• Cold starts
  • How they're impacted by VPC vs. public lambdas
  • Warming strategies (and whether they're worthwhile)
  • Strategies to minimize
• Shared code across functions
• How to manage dev / stage / prod types of multiple environments
• Lambda concurrency configuration
• Monitoring costs
• Maybe worth linking to the Well Architected Serverless Lens

Faster development server

The current make dev is passable for simple apps, but for more complex apps I the Go http server that mounts the handlers with a HTTP handler middleware of sorts.

It would be nice to add cmd/server/main.go to this project to demonstrate that pattern.

API endpoint to invoke worker

Update logs doc to use SAM CLI

The SAM CLI makes it easy to get all the logs for an app now. Should update docs to reflect

Google OAuth 2.0 "state"

A reviewer reminded me of The Most Common OAuth2 Vulnerability

How to detect, is certain OAuth implementation vulnerable?

If site doesn't send 'state' param and redirect_uri param is static and doesn't contain any random hashes - it's vulnerable.

Reviewing the implementation, there is no "state" parameter on the OAuth redirect or callback.

Digging into the Passport code, it looks like state isn’t enforced. It’s ok if it isn’t provided and also ignored if there isn’t a session store configured. I don’t have a session store configured, since that's more of an Express.js thing.

https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js#L204

Here’s the some Google guides that talks about state.

https://developers.google.com/identity/protocols/OAuth2WebServer
https://developers.google.com/identity/protocols/OpenIDConnect#state-param
https://developers.google.com/identity/protocols/OpenIDConnect#createxsrftoken

So I think we need to come up with a Lambda@Edge friendly way to set “state” and verify it….

Go Debugger Docs

Databases: Note about Serverless Aurora?

I believe all of these reasons are why they're building Amazon Aurora Serverless. It's in preview right now, but it might be worth a callout as an option in the future?

PostgreSQL, one of the goto databases for web apps, may not handle 100 simultaneous connections without adding connection pooling, or may require migrating data to a higher capacity server. Both are heavy operational tasks.

DynamoDB is better suited to this challenge.

DynamoDB not as easy to use as a developer. It lacks transactions so if we need to update multiple records atomically, our code has to handle locking, updating, then unlocking. It has a simplistic indexing model so we have to design our table keys and limited indexes carefully to avoid scanning the entire table. It's scaling model isn't perfect, so there are scenarios where DynamoDB will be inefficient and expensive at medium to large scale.

Project organization

After adding the auth javascript handler things are starting to feel a bit disorganized. Some ideas:

• Kill the checked in Go vendor directory
  • Move yamllint up into bios
• Introduce handlers/go and handlers/js hierarchy
• Move .go files into hierarchy of smaller packages, e.g. pkg/user/user.go, pkg/work/work.go

Why SAM?

Maybe interesting to compare and contrast SAM, Serverless, Apex, ... and the benefits you get from building within the native AWS ecosystem.

Auth

I plan to port an OAuth / JWT example over.

One question is how to get the OAuth redirect url. Perhaps you can generate it from the Request:

2018/03/04 18:26:03 EVENT: {Resource:/auth Path:/auth HTTPMethod:GET Headers:map[Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8 Accept-Language:en-US,en;q=0.9 Host:jkpyz7xxxx.execute-api.us-west-2.amazonaws.com X-Amzn-Trace-Id:Root=1-5a9c3a3b-90b30f21a64603017bc3891e X-Forwarded-Proto:https X-Forwarded-For:73.92.1.8, 205.251.214.101 Accept-Encoding:gzip, deflate, br CloudFront-Is-Desktop-Viewer:true CloudFront-Viewer-Country:US Via:2.0 3cc911e7eb2df956e3f7c8f27c19xxxx.cloudfront.net (CloudFront) X-Amz-Cf-Id:c-rDYArkjqKDPoUwiTCMgHEj29egDuMOiVLX-v-vLjCm_i51DYoASQ== CloudFront-Forwarded-Proto:https CloudFront-Is-SmartTV-Viewer:false CloudFront-Is-Tablet-Viewer:false User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36 CloudFront-Is-Mobile-Viewer:false upgrade-insecure-requests:1 X-Forwarded-Port:443] QueryStringParameters:map[] PathParameters:map[] StageVariables:map[] RequestContext:{AccountID:XXXXXXXXXXXX ResourceID:mh6gbm Stage:Prod RequestID:7ee1c72a-1fd9-11e8-8695-f9317416d457 Identity:{CognitoIdentityPoolID: AccountID: CognitoIdentityID: Caller: APIKey: SourceIP:73.92.1.8 CognitoAuthenticationType: CognitoAuthenticationProvider: UserArn: UserAgent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36 User:} ResourcePath:/auth Authorizer:map[] HTTPMethod:GET APIID:jkpyz7xxxx} Body: IsBase64Encoded:false}

Step Functions

Demonstrate and document step functions to automate things

New sam CLI install method

See https://github.com/awslabs/aws-sam-cli/releases/tag/v0.6.2

$ brew tap aws/tap
$ brew install aws-sam-cli

$ pip uninstall --user aws-sam-cli
$ pip3 uninstall --user aws-sam-cli

KMS Environment Encryption

Seems like we can encrypt environment variables effectively for free by adding a KmsKeyArn property to functions. See: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md

This would be an easy contribution for a first timer...

More docs

• Idiomatic AWS
• Admin and one-off tasks

DynamoDB autoscaling

It'd be interesting to demonstrate how to use DDB autoscaling to provision a 1,1 table but that could burst to 10x throughput without any management from the ops side of the table.

CloudFormation CustomResources Docs

A custom resource appeared in #67 to toggle API Gateway tracing. It would be good to document how this works with Go and Lambda.

handler -> main

It might be a bit more idiomatic to compile the binaries to main and make main.zip. Shorter too.

CodeBuild & CodePipeline

Might be nice to use CodeBuild & CodePipeline for the deployment process and have it be setup as part of the CF template.

API Gateway Authorizer

We can lift the JWTClaims helper out of our handler functions and into an API Gateway Authorizer, further reducing application code and increasing security.

Support for this in SAM is questionable. It doesn't look like a Serverless::Function supports adding an authorizer yet, but maybe you can assign then via Swagger.

• aws/serverless-application-model#248
• aws/serverless-application-model#25
• aws/serverless-application-model#49

for custom domains for regions other than us-east-1

#12 introduced a side effect where the cert has to be in us-east-1.

If the stack was created in us-west-2, etc. you get this error: Invalid certificate ARN: arn:aws:acm:us-west-2:572007530218:certificate/26e813b5-3a21-43e0-bbfc-ad02afa17cb0. Certificate must be in 'us-east-1'.

It looks like you can manually create the ACM cert in us-east-1 and assign it to an API Gateway in another region, but that defeats the automation.

We could still support us-west-2, etc, by adding a cert ARN parameter and the various conditional bits

Debugging

It would be nice to support and document debugging Go apps. However it looks like it might not work upstream yet:

aws/aws-sam-cli#281

CloudTrail

It is always nice to be able to audit KMS access. Should this stack create a CloudTrail?

Wishlist

What are the things that still feel painful to you and that you hope AWS fixes in the next 6-12 months?

watchexec + make + start-api

It would be nice if aws-sam-local start-api would pick up the latest code changes. We should be able to do this with a tool like watchexec calling make handlers on changes. Go 1.10 build caching might make this a fast enough experience.

admin and one-off tasks

Lambda poses a challenge running interactive sessions for things like database migrations, reporting, etc. It could be useful to at least document some techniques if not build a tool.

Change Sets

It might be nice to display the CloudFormation deploy change set:

Something like this but it needs to wait for the change set execute like cloudformation deploy does.

deploy: BUCKET = pkgs-$(shell aws sts get-caller-identity --output text --query 'Account')-$(AWS_REGION)
deploy:
	@aws s3api head-bucket --bucket $(BUCKET) || aws s3 mb s3://$(BUCKET) --region $(AWS_REGION)
	@aws cloudformation package --output-template-file out.yml --s3-bucket $(BUCKET) --template-file template.yml
	@aws cloudformation deploy --capabilities CAPABILITY_NAMED_IAM --no-execute-changeset --template-file out.yml --stack-name $(APP)
	@aws cloudformation describe-change-set \
		--change-set-name $$(aws cloudformation list-change-sets --stack-name gofaas --output text --query 'Summaries[0].ChangeSetName') \
		--output table \
		--query 'Changes[*].{Action:ResourceChange.Action,Resource:ResourceChange.LogicalResourceId,Type:ResourceChange.ResourceType,Replacement:ResourceChange.Replacement,Target:ResourceChange.Details[0].Target.Name}' \
		--stack-name $(APP)
	@aws cloudformation execute-change-set \
		--change-set-name $$(aws cloudformation list-change-sets --stack-name gofaas --output text --query 'Summaries[0].ChangeSetName') \
		--stack-name $(APP)
	@aws cloudformation describe-stacks --output table --query 'Stacks[*].Outputs' --stack-name $(APP)

seeding DynamoDB with data

What's the best way to seed DB with data from a json file?

• Another lambda/worker function?
• How to best make sure the function is run one-off & only if DB is not already seeded?

API Caching

Consider exploring the caching feature of API gateway:

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-caching.html

Lambda@Edge and OAuth doc tweak?

The docs about private static websites show a bucket configuration that implies usage of the website endpoint (WebsiteConfiguration):

Resources:
  WebBucket:
    Properties:
      BucketName: !Ref WebDomainName
      WebsiteConfiguration:
        ErrorDocument: 404.html
        IndexDocument: index.html
    Type: AWS::S3::Bucket

But the CloudFront distribution created there is not pointing at the website endpoint:

Origins:
          - DomainName: !Sub ${WebBucket}.s3.amazonaws.com
            Id: !Ref WebBucket
            S3OriginConfig:
              OriginAccessIdentity: !Sub origin-access-identity/cloudfront/${WebOriginAccessIdentity}

Which, if I understand it correctly, is because accessing S3 objects through the website endpoint is incompatible with the origin access identity feature that is used here.

First, this is an awesome project, thank you! I found it to be very helpful.

Second, a question: is my reading accurate that WebsiteConfiguration is not really doing anything here? If so I think it would be helpful to remove it from the template to clarify that the WebsiteConfiguration and S3 website endpoint are incompatible with locking down access to the bucket in this way.

I'm happy to send a PR if it makes sense.

Security docs

#45 was merged without docs:

• Docs - Static web security with CloudFront, Lambda@Edge and Google OAuth
• Docs - Function security with JWT
• Docs - Parameter store
• Docs - Update static sites with improved S3 origin strategy

Encryption at rest confusion

Encrypting data before saving it to the database is a security best practice called "encryption at rest".

Generally encryption at rest would be more along the lines of what AWS can do for you: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/EncryptionAtRest.html

It has limited security value (basically guarding against someone walking out of the data center with a hard drive). However it is extremely simple to use when AWS supports it and great for checking off compliance boxes ✅.

Your example would be closer to column-level encryption or something like that.

Periodic Event Rule

I don't think the period worker is wired up quite right. It seems to get triggered by all cloudwatch events, not just the related rule in the template

Use x-ray in Periodic Worker

There's a bug in the aws sdk where the s3manager isn't using context properly.

I reported it upstream so hopefully it gets addressed there.

If not, we could use S3() more directly to clean the bucket, or live without tracing.

Docs license

Explicitly license the docs with something like:

This work is copyright Noah Zoschke and licensed under a Creative Commons Attribution 4.0 Unported License.