devopsglobalsolutions/manheim-c7n-tools

Manheim's Cloud Custodian (c7n) wrapper package, policy generator, runner, and supporting tools.

manheim-c7n-tools

Manheim's Cloud Custodian (c7n) wrapper package, policy generator, runner, and supporting tools.

This project provides common tooling, distributed as a Docker image, for managing Manheim's cloud-custodian (c7n) tooling, including c7n itself, c7n_mailer, and our custom components. This project/repository is intended to be used (generally via the generated Docker image) alongside a configuration repository of a specific layout, containing configuration for one or more AWS accounts.

• Full Documentation: https://manheim-c7n-tools.readthedocs.io/
• TravisCI Builds: https://travis-ci.com/github/manheim/manheim-c7n-tools
• Docker image: https://hub.docker.com/r/manheim/manheim-c7n-tools

For documentation on the upstream cloud-custodian project, please see https://cloudcustodian.io/docs/index.html and the source code at https://github.com/cloud-custodian/cloud-custodian.

Introduction and Goals

Cloud Custodian (a.k.a. c7n) is a flexible rules engine for reporting on and enforcing policy in AWS. Manheim has migrated from Netflix Janitor Monkey to Cloud Custodian for our tag enforcement, resource cleanup, cost reduction, and other policy needs. This project provides common tooling to allow us to deploy and manage c7n across multiple AWS accounts.

We're currently deploying all Cloud Custodian policies via AWS Lambda. The default is to run policies once per day, but other execution triggers are available including arbitrary timer triggers as well as CloudTrail, AWS Config and CloudWatch Events.

A description of the initial tooling that turned into this project can be found in this blog post.

.. _index.main_components:

Main Components

The following commands are available in the Docker container (or Python installation), generated as Python package entrypoints:

• manheim-c7n-runner - A single entrypoint to wrap running one or more, or all, of the following steps (as well as custodian itself, c7n-mailer deploy, and Sphinx documentation build) in the proper order, in either run (real) or dryrun mode.
• policygen - The python script to generate the actual custodian YML config files from a configuration repo/directory, as well as handling layering of mailer template files. Must be run from a config repository directory.
• s3-archiver - Script to clean up custodian S3 buckets by moving logs from any deleted policies to an "archived-logs/" prefix.
• dryrun-diff - Script to compare the number of resources matched per-policy, per-region between a dryrun and the last actual run of each policy, and write the results to a Markdown file (to be added as a comment on the PR). Optionally, a HTML report listing the affected resources in the dryrun can be generated when a jinja template is provided in the ./reporting-template directory.
• errorscan - Script using boto3 to examine CloudWatch Metrics, Logs, and SQS Dead Letter Queue for cloud-custodian Lambda functions, and alert on any failed executions, dead letters, etc.
• c7n's built-in mugc Lambda garbage collection. This is vendored-in to manheim-c7n-tools, as c7n provides it only as a non-executable Python source file in their git repo.
• c7n's c7n-mailer installed as a dependency for convenience.

Installation and Usage

See Installation and Usage