jantman/awslimitchecker

A script and python package to check your AWS service limits and usage via boto3.

awslimitchecker

Master:

Develop:

A script and python module to check your AWS service limits and usage, and warn when usage approaches limits.

Users building out scalable services in Amazon AWS often run into AWS' service limits - often at the least convenient time (i.e. mid-deploy or when autoscaling fails). Amazon's Trusted Advisor can help this, but even the version that comes with Business and Enterprise support only monitors a small subset of AWS limits and only alerts weekly. The new Service Quotas service can also help with this, but relies on CloudWatch alarms per-limit to notify you when you approach your limits; this cannot easily scale to the hundreds of current service limits. awslimitchecker provides a command line script and reusable Python package that queries your current usage of AWS resources and compares it to limits (hard-coded AWS defaults that you can override, API-based limits where available, Service Quotas data where available, or data from Trusted Advisor where available), notifying you when you are approaching or at your limits.

Full project documentation for the latest release is available at http://awslimitchecker.readthedocs.io/en/latest/.

Status

awslimitchecker is mature software, with approximately 13,000 downloads per month and in daily use at numerous organizations.

Development status is being tracked on a board at waffle.io: https://waffle.io/jantman/awslimitchecker

What It Does

• Check current AWS resource usage against AWS Service Limits
• Show and inspect current usage
• Override default Service Limits (for accounts with increased limits)
• Compare current usage to limits; return information about limits that exceed thresholds, and (CLI wrapper) exit non-0 if thresholds are exceeded
• Define custom thresholds per-limit
• where possible, pull current limits from Trusted Advisor API
• where possible, pull current limits from each service's API (for services that provide this information)
• where possible, pull current limits from the Service Quotas service
• Supports explicitly setting the AWS region
• Supports using STS to assume roles in other accounts, including using external_id.
• Optionally refresh Trusted Advisor "Service Limits" check before polling Trusted Advisor data, and optionally wait for the refresh to complete (up to an optional maximum time limit). See Getting Started - Trusted Advisor for more information.
• Optionally send current usage and limit metrics to a metrics store, such as Datadog.
• Optionally send warning/critical alerts to notification providers, such as PagerDuty.

Requirements

Either Docker in order to run via the docker image, or:

• Python 3.5 or newer.
• Python VirtualEnv and pip (recommended installation method; your OS/distribution should have packages for these)
• boto3 >= 1.4.6 and its dependency botocore >= 1.6.0.

Alternatively, awslimitchecker may be installed through conda with conda install -c conda-forge awslimitchecker. This will automatically install the required dependencies as well.

Installation and Usage

See Getting Started.

Credentials

See Credentials.

Getting Help and Asking Questions

See Getting Help.

For paid support and development options, please see the Enterprise Support Agreements and Contract Development section of the documentation.

There is also a gitter.im chat channel for support and discussion.

Changelog

See Changelog.

Contributions

Pull requests are most definitely welcome. Please cut them against the develop branch. For more information, see the development documentation. I'm also happy to accept contributions in the form of bug reports, feature requests, testing, etc.

License

awslimitchecker is licensed under the GNU Affero General Public License, version 3 or later. This shouldn't be much of a concern to most people; see Development / AGPL for more information.

awslimitchecker's Issues

Full and complete docs

Perform complete docs review, document stuff.

EC2 "Reserved Instances - purchase limit (monthly)" limit

TA reports the "Reserved Instances - purchase limit (monthly)" EC2 limit, but ALC doesn't know about that limit yet.

setup hosted services

1. cut a 'develop' branch, start doing work there
2. Open the repo
3. Setup badges and ancillary services:
   1. RTD
   2. Travis CI
   3. landscape.io
   4. codecov.io
4. Make sure that badges and ancillary services work, including RTD.
5. Blog post and tweet.

Handle API requests with markers

For boto requests that have max_results and return a marker, we need to handle iterating the marker. The most logical way to do this is probably a utility function that takes the boto connection object, the method name, and a dict of args, and returns the combined responses.

Support s3 service

As far as I can find, the only limit on buckets is the 100 total buckets per account.

Need to make sure that if we're checking this, we're looking at cross-region (is this easily possible?)

Autoscaling Service

implement it

Fixes suggested by landscape.io

https://landscape.io/github/jantman/awslimitchecker

runner dump data in JSON

runner should have an option to return output - at least services, limits, usage, check results - in JSON

Docs about regions

Add documentation on regions - currently only one at a time, and from boto config.

runner limit overrides

Requires #9
Expose ability to set limit overrides in runner

ToDos

Grep -i source and docs for "todo" and "tbd"

Source notices

Ensure our docs on AGPL make clear the requirements in section 5b and section 13.

Limits should take Service class reference

Instead of storing the service name, AwsLimits should take a reference to an _AwsService object.

AGPL docs

Since many people may be unfamiliar with it, add some docs on AGPL, what it means to you, how to be compliant, etc.

Version and URL information

Fill out project_url and version to find the actually-installed version and source, even if it's from package vs git.

Integration tests fail for Pull Requests in Travis

See e.g. https://travis-ci.org/jantman/awslimitchecker/jobs/72610593#L711

When Travis checks out a pull request, it checks out the merge of the PR to master, i.e. git fetch origin +refs/pull/45/merge. The git-based integration tests (Test_AGPLVersionChecker_Acceptance.test_install_git_e_dirty, Test_AGPLVersionChecker_Acceptance.test_install_git_e and Test_AGPLVersionChecker_Acceptance.test_install_git) attempt to install awslimitchecker from GitHub using the commit hash of the current git checkout that the tests exist in - which, in this case, is the merge hash, which doesn't exist yet in the repo.

Options that I can think of:
(1) don't just check out whatever hash the test repo is at, if it doesn't exist in origin.
(2) Look at the value of the TRAVIS_PULL_REQUEST environment variable, and do something different in the integration tests if we're testing a PR (such as installing +refs/pull/$TRAVIS_PULL_REQUEST/merge instead of the SHA).

Trusted Advisor - option to refresh and wait on all limit-related checks that haven't been refreshed in time

option to refresh and wait on all limit-related checks that haven't been refreshed in X seconds/minutes

--list-defaults actually lists effective limits

the runner command --list-defaults says it lists the defaults, but it actually lists the effective (overridden) limits. Fix this.

Integration tests

Each function or class that actually interacts with the AWS API should have integration tests. They should run in a fake $HOME to prevent any ~/.boto / ~/.aws credentials from being used. The tests should require AWS credentials exported as env vars, and should check usage. Maybe we just want one overall integration test that runs checker(). We want to catch (a) any bad boto or AWS calls that raise exceptions, (b) any IAM permissions errors, and (c) any limits that didn't get their usage checked. The test should also confirm that the IAM policy for the user it's running under matches what awslimitchecker wants.

Trusted Advisor on accounts without Support

polling the TA checks on a basic account without Support throws an exception. Handle this, and log a warning. Confirm that this is correct per AWS docs (shouldn't even free accounts have access to a limited subset of checks?)

0.1.0 release

cut 0.1.0 release

Implement ElastiCache service

ElastiCache

doc checks and cleanup

get sphinx linkcheck to work, re-enable -n and -W

Travis builds fail for new releases

Travis builds of master fail when a release is done, because the commit is tagged and versionchecker picks up the tag instead of the hash

see e.g. https://travis-ci.org/jantman/awslimitchecker/jobs/72649103#L778

All tasks for specific service

Ensure we can perform all tasks for just one specific service. This includes TA.

No module named services

Tried the install directions per the README on Ubuntu 14.04 and Amazon Linux. I was not able to install the module from pip directly so I attempted to install it via source with a 'python setup.py install'. When trying to run the binary I get an error "ImportError: No module named services" using python 2.7.

Any chance this will get back into the pip library? Or is there some other issue when I am installing this?

root@ip-10-0-4-112:~# /usr/local/bin/awslimitchecker
Traceback (most recent call last):
File "/usr/local/bin/awslimitchecker", line 9, in
load_entry_point('awslimitchecker==0.1.0', 'console_scripts', 'awslimitchecker')()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 351, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2363, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2088, in load
entry = import(self.module_name, globals(),globals(), ['name'])
File "build/bdist.linux-x86_64/egg/awslimitchecker/runner.py", line 46, in
File "build/bdist.linux-x86_64/egg/awslimitchecker/checker.py", line 40, in
ImportError: No module named services

clean up badges, downloads and version badges broken

Clean up badges in README.md and docs/source/index.rst. Also put master- and develop-specific ones on separate lines.

Also, the crate.io PyPi Downloads and PyPi Version badges are broken. shields.io has supposedly-working ones at:

per month downloads: https://img.shields.io/pypi/dm/awslimitchecker.svg
version: https://img.shields.io/pypi/v/awslimitchecker.svg

Docs - runner examples

Add some examples of using the runner.

Move EBS-related limits to EBS service

Trusted Advisor lists EBS-related limits as their own service, even though they appear under EC2 in the AWS docs. Move them so they play nice with TA.

AGPL source handling

• implement source discovery, the best we can, for AGPL compliance. We should test to see if it's installed from a package or git. If from a package, point to either GitHub or PyPi. If from git, point to the repo url. In #6 instruct that anyone modifying the source must ensure this function works.

Implement RDS service

and TA checks

Implement ELB Service

and confirm TA checks

runner.py column output

runner output should be in nice columns

option to not use TA at all

option to not use TA at all

Verify IAM permissions

Verify that all checks work correctly using an IAM user with IAM permissions matching what ALC generates.

Add support for IAM limits

see http://docs.aws.amazon.com/IAM/latest/UserGuide/limits.html

The IAM GetAccountSummary API action and aws iam get-account-summary appear to provide both quota and usage information for at least some of these, so this should be simpler than most.

Finish EC2 service

Finish implementation of EC2 service

Trusted Advisor

Attempt to pull limits from Trusted Advisor where possible.

This also needs to include updating docs:

• overall docs on TA
• development - adding services/limits, adding TA checks
• CLI examples
• Generated TA info, and comment on limits where they pull from TA

Runner colorized output

Add colorized output to runner, with option to turn it off.

Logging

Ensure logging is being done in all logical places. Adjust runner logging format (maybe based on verbosity?)

Trusted Advisor - option to refresh all limit-related checks but not wait

option to refresh all limit-related checks but not wait (i.e. trigger refreshes that we'll see in the future)

Threshold overrides

• checker needs ability to set threshold overrides per-limit or as a dict, either percentage or integer
• expose this in runner as either per-limit options or a dict

Add support for spot instance

Just came across this useful tool. Can you please tell me if you plan to add support for reporting on spot instances when running awslimitchecker -u? Most of our instances are spot and this would help very much. Thanks for considering this.

skeleton file / command for new services

Create a skeleton file and/or helper script for adding new services.

Fix AGPL warning

Fix the AGPL message on init. How can this be done nicer? Maybe just print, or to stderr?

Trusted Advisor - option to refresh all limit-related checks and wait on them

option to refresh all limit-related checks and wait on them

checker.set_limit_override

Ability to set an override for a single limit instead of a dict.

Docs - python examples

Include some python examples.

--help should point to RTD

Runner --help should point to RTD

VPC Service

Implement VPC service