xandkar/beam_stats

Collect detailed Erlang VM metrics and send to arbitrary, plugable backends (StatsD and Graphite backends included).

beam_stats's Issues

Aggregate anonymous process data in StatsD and Graphite consumers

In Whisper, short-lived procs result in lots of files containing few data points and lots of disk-space-wasting nulls. We'll loose granularity, but it is the only feasible way to get at least some per-process visibility, since short-lived, dynamically-generated paths just don't scale in Whisper.

Support chunking in Graphite backend

Just like in the StatsD one

And enable/disable per-ets-table stats config

Blows up Graphite disk usage when table names are dynamically generated. Bad practice, but encountered in the wild, so have to deal with it...

Factor-out the idea of periodic (produce -> package -> transport) as freight line library

Ostensibly:

-spec produce() -> A.

-spec package(A) -> B.

-spec transport(B) -> {}.

Consider accumulating StatsD messages in the process mailbox, rather than concatenating lists

With per-process metrics added, all these concatenations may get expensive. Will see how it looks in the next few days...

Collect number of ETS tables per name

Test the full produce/consume cycle, not just consume

Aggregate ETS table data: rollup all tables with the same name

Bad format in log messages

such as: https://github.com/ibnfirnas/beam_stats/blob/cdcb989e330d3a018b44b154c190c33b9b27040c/src/beam_stats_consumer_statsd.erl#L113-L116

Handle undefined value from erlang:process_info/2

Add a process-provenance-graph consumer

The output can be a stream of edges, like:

host,time,parent,child

which can then be grouped by time to create graph snapshots at given times, which then can even be animated.

Collect reductions deltas per process

Implement generic deltas server

Which stores previous totals and produces deltas between calls.

Collect ports stats

Abstract graphite message and graphite to statsd message transformations

For improved testability and code-reuse between Graphite and StatsD consumers

Collect number of processes per origin