A toy project that periodically scrapes AIDS Walk Wisconsin fundraising data for a team and notifies you of your standing within it.
This code does what it needs to do. That doesn't include being taken very seriously.
Broadly speaking, there are three things that the code in this repo does:
- Scrape a team's fundraising data, returning a list of team members and the amounts they've raised in descending order by amount.
- Store data from each scrape in a database, so you can Have Fun With It later.
- Send an email telling you how much you've raised and where you are in the standings.
Each of these features is implemented within an AWS Lambda function.
This project also defines the infrastructure for deploying these functions to AWS. Terraform is our infrastructure provisioning tool of choice.
Beyond Lambda, this project is almost entirely dependent on AWS services. It uses DynamoDB as its database solution (not that its use cases are really exercising it) and Simple Email Service to send notifications. You will need to verify any email addresses you use.
This project is largely divided into four top-level directories:
script/includes the initialization and deployment scripts you'll need to work with the project.script/init.shis the first thing you'll want to run to set up a development environment. It will install the development dependencies and seed a.tfvarsfile that you can use for testing.script/package.shwill create a.zipartifact for the Lambda functions. You can execute this yourself, or you can call it indirectlyscript/build-and-deploy.shscript/build-and-deploy.shwill, when given the name of an infrastructure deployment (testorprod), refresh the Lambda function artifact and redeploy the infrastructure.
src/includes the code for the Lambda function handlers.tests/includes the (basic!) unit tests for the handlers. These are not included in the build artifact.infra/includes the Terraform infrastructure definitions for "test" and "prod" stacks. These both rely on shared infrastructure modules, and are separated from each other largely to make it easy to maintain state separately.
- Python 3.8
- Terraform 0.15.3 (specified in
.terraform-versionfor use with tfenv)
To set the project up for local development:
$ script/init.shTo run the tests:
$ pytestTo update the test snapshots:
$ pytest --snapshot-updateTo deploy test infrastructure:
$ script/build-and-deploy.sh testTo deploy production infrastructure:
$ script/build-and-deploy.sh prodThe infrastructure deployments are parameterized by a few settings that most deployments would want to change. These are given in the production and test infrastructure definitions.
To get you started, the test infrastructure has an example terraform.tfvars file.
script/init.sh will copy this to infra/test/terraform.tfvars for you to edit.
This project contains two Pip requirements files:
requirements.txtdefines the dependencies that are needed to run the application code on AWS Lambda infrastructurerequirements-dev.txtdefines the dependencies that are needed to develop and test the application locally
If you use an IDE, you will likely need to configure it to use requirements-dev.txt as your requirements file to have proper navigation and autocompletion.
For example, you can find instructions for how to do this in PyCharm here.
This project is not designed to or expected to last very long. This impacts most of the engineering decisions:
- AWS Terraform providers are used directly with little to no abstraction.
- The unit tests are relatively sparse, mostly serving to flag accidental regressions during local refactoring. The project was not created with test-driven development.
- The Python source code is not modularized or packaged into separate artifacts for each Lambda function.
- There is no infrastructure definition for a deployment pipeline.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent