capablemonkey/nitpicker

functional service monitoring for high transparency

nitpicker's Issues

Collect data to render previous 7 days and 30 days graphs

From a set of, say, 50,000 response time measures, collected every 10 seconds over a period of 30 days, we need to be able to reduce that data set to a smaller one, say 100 items, which can be cheaply sent to the Meteor browser client and rendered to a graph.

Ideally, we would be able to collect this data directly from MongoDB without having to pull it into Meteor and comb it.

How best to reduce this dataset from 50k to 100?

Possible routes:

1. Retrieve 100 evenly spaced results from the set; i.e. every n-th element from the set.
2. Split the 50k set into 1/100th slices, and average each slice.
3. Find the average response time for each day, and plot those ~31 points.

Another consideration is whether we can afford to do this calculation for each client viewing the site. We will probably need to cache this.

Maybe, just calculate the average response time for each day, and store that as a document in a separate collection: AverageResponseTimePerDay. Then, every time we need to graph this, send over the average for the past 30 days.

We can set up a cron job to calculate the average response time for a day at the end of each day.