aleksandr-filichkin/aws-lambda-runtimes-performance

AWS Lambda Performance comparison

AWS Lambda Performance comparison for all runtime

This repo contains full code and all scripts for the most popular AWS Lambda flow:

The full analyze is here https://filia-aleks.medium.com/benchmarking-all-aws-lambda-runtimes-in-2021-cold-start-part-1-e4146fe89385

Supported runtimes x86_64 and ARM

• Java 11
• NodeJs 14.x
• Python 3.9
• Go 1.17.7
• Ruby 2.7
• .Net 3.1
• Custom runtime with GraalVM 22.0
• Custom runtime with Rust 1.58.1

How to build project

The project has CI/CD using GitHub Actions, please check GitHub workflow

To build it manually you just need

• Docker
• SAM
• AWS account with S3 bucket in us-east-1 Region (update existingS3Bucket with your bucket value)
• run build.sh from the root

Contribution

I’m not an expert in all these languages and I’m happy to see MR in the repo with performance improvements.

I’m going to support this repo and run the performance test every 3 months. I believe in opensource collaboration :)

aws-lambda-runtimes-performance's Issues

Add or upgrade to Node 18

AWS has launched their Node 18 runtime along with AWS SDK 3. It would be great to have some numbers on Node 18 performance, as it offers a lot of improvements over 14.x.

https://aws.amazon.com/blogs/compute/node-js-18-x-runtime-now-available-in-aws-lambda/

Thanks for this great project!

.NET 6 +

I would suggest the same test with .NET 6.x

Make Rust Lambda Use Single-Threaded Runtime

A multithreaded only adds overhead with no technical benefit in an AWS Lambda environment, where lambdas are invoked without per-instance concurrency or parallelism.

Reference to original note.

Reference to relevant tokio docs.

Cold Start performance of Node.js runtime can be improved

Requiring aws-sdk package (which is AWS SDK V2) will load entire AWS Service Clients (e.g. CloudWatch, EC2, ECS, Kinesis, S3, ... etc.), not only DynamoDB Client.

You will see better start-up performance if you load DynamoDB client only (e.g. const DynamoDB = require('aws-sdk/clients/dynamodb').

Also, AWS Javascript SDK V3 is currently in GA. AWS Javascript SDK V3 is modular, and can be optimized using bundlers to eliminate dead codes, reducing its size, and also reduces startup-performance (Also it's faster than using individual service client!). See https://aws.amazon.com/ko/blogs/developer/modular-packages-in-aws-sdk-for-javascript/ for details.

Suggest NodeJS TLS Connection Reuse in AWS SDK

Nice to see others profiling runtimes.

I ran into this issue with AWS Lambdas written in NodeJS being bizarrely slow a while back. It’s a legacy issue with how the AWS SDK for JavaScript works. If you have time, I suggest you try again with this one simple fix. I think you’ll be impressed.

https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/node-reusing-connections.html

You can make the Go lambda faster.

If you compile the golang example with go build -tags lambda.norpc you can deploy the go version, without changes, to a "lambda custom runtime" and you will create a lambda that doesn't have amazon linux 2 in the image, making it smaller and faster to load.

https://github.com/aws/aws-lambda-go/blob/main/lambda/entry.go#L64-L66

AMD64 vs ARM?

Do you have any intention of running these benchmarks with ARM processors?