crccheck/cloudwatch-to-graphite

Helper for pushing AWS CloudWatch metrics to Graphite

cloudwatch-to-graphite's Issues

Python 2.7+ requirement should be documented in Readme.

Will not work w/ Python 2.6.

Add utility for generating config files

would be helpful. I think it should be called plumbum to go with the leadbutt theme.

Help Wanted: maintainers wanted

I, @crccheck, don't use graphite these days. So I don't have the personal need to maintain this project anymore.

Ideas:

• move this to a possibly new organization
• add more people to the pypi project
• add more people with push access

There are some basic levels of code quality I would like this project to meet that I would have put in (PR checks, contributers.md, code coverage, github templates) but I haven't had the time to do it yet.

Doc: Provide a more involved config file (example provided)

The provided sample config file is little to simple to be useful, especially for those not too familiar with YAML. Here is a more complex one that avoids duplication via YAML anchor+alias. It would be helpful to include some of this, I believe.

# Define defaults that can be referenced below
My_Defaults:
  - &ElbMetric
    Namespace: "AWS/ELB"
    Dimensions:
      # You can have multiple dimensions, but boto will only return the last one
      LoadBalancerName: "awseb-e-v-AWSEBLoa-ZELK76TU0GP"
Auth:
  region: "us-west-1"
Metrics:
- <<: *ElbMetric
  MetricName: "RequestCount"
  Statistics: "Sum"
  Unit: "Count"
# You can list additional metrics in one file. Just be careful about rate limits.
- <<: *ElbMetric
  MetricName: "UnHealthyHostCount"
  Statistics: "Maximum"
  Unit: "Count"
# You can list additional metrics in one file. Just be careful about rate limits.
- <<: *ElbMetric
  MetricName: "Latency"
  Statistics:
  - "Average"
  - "Maximum"
  Unit: "Seconds"
- <<: *ElbMetric
  MetricName: "HTTPCode_ELB_5XX"
  Statistics: "Sum"
  Unit: "Count"
# OPTIONAL: set defaults for all metrics in this file
Options:
  Count: 1 # How many Periods to return (note: there is a max num datapoints you can get at once)
  Period: 5 # [min], minimum 1

Thank you!

s3 support

I saw the PR for s3/redshift support that was closed by the author, was wondering if this will be added in some way?

beanstalk support

Elastic beanstalk presents challenges because the resources are extremely ephemeral. I have a work-in-progress solution in development; that is viewable as a compare here: master...krux:pkrohn-EPE-58-elastic-beanstalk

Feedback appreciated.

Add 'options' config to the root level of the yaml

and refactor option loading (it needs it!) so the priority is: command line, metric, config, global default

Document hosted graphite usage

No need to use a custom formatter to adapt to use a hosted graphite solution. Here's an example for hostedgraphite:

$ leadbutt | awk '{print "123-456-789-0."$0}' | nc carbon.hostedgraphite.com 2003

Upgrade to Boto3

Boto 2 (python package boto) cannot filter ELBs by tags, this is hopefully fixed in v3.

See boto/boto#2549

Thank you!

plumbum connections hard-coded to 'us-east-1'

Are there plans for plumbum to support other regions? How should one handle the hard-coded connection?

Need a more helpful message when config file is missing

it throws the IOError right now.

Wish there was a way to explore the template context to help write a template

in plumbum, when writing a new template, you're not sure how exactly to get at the juicy bits. is it resource.id or resource.name, is it in a tag? are tags even available?

I think if a template isn't specified, it should drop you into an interactive mode. Introspecting and displaying what's available would be nice, but would get messy fast.

Version numbers in 3 places

There are version numbers in 3 places, I intend to fix this by refactoring into a module structure, and just want to be sure @crccheck and @tmclaugh are aware.

no tests for filters, which are broken

I think I broke filtering when I switched from docopt to argparse for plumbum. The filter fix is pretty simple, and a test to prevent repeating this breakage would be great.

Support for EBS

Support getting cloudwatch metrics for EBS volumes

add ability to read auth from the config file

would be helpful for running this in cron jobs because setting up the environment for that is always a pain

two argument parsers in use

It's my fault, I introduced the second one. I'll refactor to consolidate on argparse, since typing of some args is an issue. cc @tmclaugh and @crccheck.

Should period be in minutes or seconds?

It's supposed to be seconds, but since they're limited to being multiples of 60, I figure minutes is human friendlier. But juggling two separate systems of numbers is annoying.

Finalize YAML schema

metric vs Metric is the big thing.
AWS's cloudwatch seems to use TitleCase for everything.

Makefile broken on OS X

The makefile should detect that it is running on OS X and adjust the options passed in to sed accordingly, so if does not make a bad change, then commit it.

See 025a9ac.

how filter works?

Could you please provide example of how plumbum filter works? I wasn't able to figure out how it work and receiving errors all the time.

Traceback (most recent call last):
  File "/home/vagrant/.pyenv/versions/2.7.6/bin/plumbum", line 11, in <module>
    sys.exit(main())
  File "/home/vagrant/.pyenv/versions/2.7.6/lib/python2.7/site-packages/plumbum.py", line 222, in main
    resources = list_resources[namespace](region, filters)
  File "/home/vagrant/.pyenv/versions/2.7.6/lib/python2.7/site-packages/plumbum.py", line 136, in list_elb
    return lookup(instances, filter_by=filter_by_kwargs)
  File "/home/vagrant/.pyenv/versions/2.7.6/lib/python2.7/site-packages/plumbum.py", line 82, in lookup
    return list(filter(filter_key(filter_by), instances))
  File "/home/vagrant/.pyenv/versions/2.7.6/lib/python2.7/site-packages/plumbum.py", line 76, in filter_instance
    for key, value in filter_args.items()])
AttributeError: 'str' object has no attribute 'items'

metrics order is not chronological

does it matter? it all gets fed into graphite which handles it fine.