alexdlaird/air-quality-bot

Texting service to receive current air quality conditions and maps, powered by AirNow, Twilio, and AWS

The Air Quality Bot is generally available by texting a zip code (and optionally the word "map") to [phone]. The bot will respond with the latest air quality report for your region.

The instructions below illustrate how to similarly setup the bot in your own AWS and Twilio environments.

Getting Started

AWS Initial Setup

Create a new Role from a Policy with the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "cloudwatch:PutMetricData",
                "dynamodb:CreateTable",
                "dynamodb:GetItem",
                "dynamodb:PutItem",
                "dynamodb:UpdateItem",
                "dynamodb:DescribeTable",
                "dynamodb:GetShardIterator",
                "dynamodb:GetRecords",
                "dynamodb:ListStreams",
                "dynamodb:Query",
                "dynamodb:Scan"
            ],
            "Resource": "*"
        }
    ]
}

Install and configure the AWS CLI for the same account for which the Role was created.

AWS Lambdas

Initialize the deployment environment by running make install locally, then edit the .env file's AWS_ROLE with the ID of the Role created above and the AIRNOW_API_KEYS list with one or more AirNow API keys.

Note that, on initial deploy, your Lambdas will be pointing to the an endpoint that does not exist until you complete the AWS API Gateway Routes section below and update the AIR_QUALITY_API_URL variable to point to the deployed endpoint.

Deploy the Lambdas to your AWS environment using the deploy script:

./deploy.sh

Optionally, TTL for ZipCode fields in the DynamoDB table can be enabled by going to the AWS console and enabling TTL on the TTL field.

AWS API Gateway Routes

Create an API Gateway. In the API, do the following:

• Create a new "Resource" with a path of /inbound
  • Create a new "POST" method with the "Integration type" of "Lambda Function" and point it to the Lambda AirQuality_inbound_POST
    • Edit the "POST" method's "Integration Request"
      • Under "Mapping Templates", add a "Content-Type" of application/x-www-form-urlencoded using the "General template" of "Method Request Passthrough"
    • Edit the "POST" method's "Method Response"
    • Edit the 200 response so it has a "Content type" of application/xml

Last, under the "Integration Response" for /inbound, edit the 200 response. Under "Mapping Templates" of "Content-Type" of application/xml with the following template:

#set($inputRoot = $input.path('$'))
$inputRoot.body

Additionally, create the following "Resource" paths:

• /aqi

Under each of the above, do the following:

• Create a new "GET" method with the "Integration type" of "Lambda Function" and point it to the Lambda AirQuality_<ROUTE_NAME>_GET, where <ROUTE_NAME> corresponds to the name of the Lambda we created to execute on this method
  • Edit the "GET" method's "Method Request"
    • Change the "Request Validator" to "Validate query string parameters and header"
    • Add a required "URL Query String Parameter" of zipCode
    • Edit the "GET" method's "Integration Request" - Under "Mapping Templates", add a "Content-Type" of application/json using the "General template" of "Method Request Passthrough"

Deploy the new API Gateway. Note the newly generated Invoke URL and update the AIR_QUALITY_API_URL variable in .env, then redeploy your Lambdas by running ./deploy.sh again.

Setup Twilio

In Twilio, create a phone number and set it up. Under "Messaging", select "Webhook" for when "A Message Comes In", select "POST", and enter the deployed API Gateway URL for /inbound.

That's it! Your bot is now setup and ready to respond to texts.

Local Development

To test the bot locally, simple Flask server is included in devserver.py. Ensure proper dev values are in .env.dev, then execute:

make run-devserver

This will start a server with the appropriate routes and in-memory datastores, and it will also start a ngrok tunnel using pyngrok so the bot can be fully tested end-to-end.

CI Build

If you would like the project to build for you in a CI system, you may need to add the following environment variables to the CI's console. Their actual values do not matter (use dummy values, not real), but certain versions of the boto dependency need them to be present when initializing its configuration.

• AWS_ACCESS_KEY_ID
• AWS_DEFAULT_REGION
• AWS_SECRET_ACCESS_KEY

Deploy Updates

After the initial installation in to your AWS environment, updates to the Lambdas can easily be redeployed at any time by rerunning the deploy script:

./deploy.sh

Contributing

If you would like to get involved, be sure to review the Contribution Guide.

Want to contribute financially? If you've found Air Quality Bot useful, sponsorship would also be greatly appreciated!

air-quality-bot's Issues

Fix Travis build

Build is failing preliminarily on dependencies. Works locally, so must be a config issue with Travis that needs to be worked out.

When an AirNow Request Fails, Retry After 30 Seconds

AirNow requests fail for two reasons: AirNow is legitimately down, or AirNow experienced a hiccup. If AirNow is legitimately down, retrying in 30 seconds will do no real harm anyway, but when it's a hiccup and the user sends a second text, it almost always goes through.

To alleviate this need for the user to send a new text, simply queue a retry (and return from the Lambda without sending a text response), then retry the same request to AirNow after 30 seconds. If the second request fails, then tell the user AirNow seems down, otherwise give them the data they requested.

Have Dynamo ZipCode records expire after 24 hours

Currently, if AirNow fails to give us an AQI within the timeout, and a cached value is available from a previous request, we will fall back to the cached value. This is fine if the cached value is only a couple hours old, but if the cached value is older than 24 hours, this gets confusing (as a date is not given, and the data may be misleading at that point).

The best way to fix this is to add a TTL field to the table and continually push it out 24 since the last update.

Once this TTL field is added, configure Dynamo to use it to expire records.

Update README to reflect Dynamo configuration.

Stabilize `make test-e2e` command for Travis CI

Improvements to deploy.sh

• Add a .env.example file, create a Makefile that copies sample values in to a .env. When deploy.sh is run, utilize values in the .env file for environment variables in the Lambda configuration. Update instructions then from the README to more simply say the .env file should be updated with proper configurations before a deploy.
• Check if the Dynamo DB table already exists. If it does not, create it. Remove instructions then from the README.
• Check if the Lambda functions already exist. If they do not, create them first using the CLI.

When DyanmoDB tables are created from the CLI, the TTL can not be set, so the README should be updated to reflect manually (but optionally) setting this.

Use Direct S3 URLs for AirNow Maps

AirNow map URLs typically resolve to something from http://files.airnowtech.org, but AirNow's static files are served from S3. Occasionally, during high load, AirNow's files subdomain fails to resolve. See if we can't update these mappings to go directly to the S3 bucket.

Improve ReportingAreas "CachedAQI"

Could use some better validation around falling back to the ReportingArea's "CachedAQI" value if the ZipCode has old cached data but the ReportingArea has more recent data. For example, if the ReportingArea's "CachedAQI" is over 24 hours old, it shouldn't be used (and perhaps should be expired).

Create local tests

The current tests can only be run against the deployed Lambda. Create local tests to execute the code in a dev box before deploying (and also perhaps a CI).

Add "evacuate" keyword to see if zip code is in evacuation zone

The requesting phone number should then also be stored for up to 30 days, and if the evacuation status changes should be proactively texted to alert the user of the new evacuation status.

Update bot's standard response to tell users about this keyword.

Add "fire" keyword for fire map

When a user requests a zip code with the word "fire", return a region map (same dimensions as AirNow's region map for that zip) of Google's Crisis Map with wildfires highlighted.

Update bot's standard response to tell users about this keyword.

Properly handle when AirNow responds with an error

When AirNow is not overloaded but responds with an error, it gives us XML (even if JSON is requested). This causes an exception, as we try to parse the JSON, and the subsequent error causes the bot to suggest AirNow is unavailable for the area (when it should say an error occurred).

A test could easily be written to mock this response and return a proper error.

Add Metrics Integration (CloudWatch and/or Datadog)

https://docs.datadoghq.com/integrations/amazon_lambda/