linktdp/sqs-worker-serverless

Example for SQS Worker in AWS Lambda using Serverless

⚡️ SQS Worker with Serverless

Experiment for an Amazon SQS worker with AWS Lambda using lawos and serverless.

More details: Serverless Amazon SQS Worker with AWS Lambda

Setup

• SQS Queue with your messages
• SNS Topic to handle CloudWatch Alarms
• DynamoDB table to persist configuration
• CloudWatch Schedule as cron replacement
• Three (scale, worker, process) AWS Lambda functions

Workflow

• CloudWatch Alarms on queue length post to SNS
• SNS Topic triggers scale Lambda function
• Function scale updates configuration in DynamoDB
• CloudWatch Schedule invokes worker every x minute(s)
• Function worker reads configuration from DynamoDB
• Function worker invokes process function(s)

Auto-Scaling with CloudWatch Alerts

Workers with CloudWatch Schedule

Deploy

$ > yarn install
$ > yarn deploy

Add noise to SQS

You should have some data in your queue to test this setup. Use wrk to send messages to SQS, but make sure to enable anonymous access to sendMessage for your queue first!

$ > wrk -c35 -d60 -t35 \
    -s helpers/wrk.lua \
    https://sqs.REGION.amazonaws.com/ACCOUNT-ID/YourQueueName