Allow counter increments over TSDs

[opentsdb]

Allow counter increments over TSDs

[marcuswestin]

+1

Rather than having an interval between data points, could the new rpc simply update the value for a metric at the requested ts?

E.g. if in addition to "put " we could "increment ", and have that simply overwrite the previous of at with <value1 + value>.

Perhaps that's exactly what you were saying :)

[scottcarey]

Rather than have OpenTSDB try and increment a value, why not just store the event count over the interval?

Well, I tried this, and all of the aggregation functions break, and the inverse of the 'Rate' checkbox would be needed to see a total.

e.g. metric is event counts during a time interval -
host: [time1, time2, time3]
a: [2, , 1]
b: [1, 1, ]

sum: [3, 2.5, 2]

The problem is that OpenTSDB is inventing data points for time 2 and 3 where there is no value as interpolations between the neighboring values. This makes the sum across hosts completely wrong for this use case, unless I populate all of the empty spots with 0 values, which for my data (very sparse) multiplies the total data stored several fold.

In short, all of OpenTSDB's metrics are treated as "measurements" of an underlying continuous function, and its aggregate functions invent data by interpolation.

There (currently) is no mode to treat metrics as discrete values such that the above sums are as would be required for this use case:

sum-discrete: [3, 1, 1]

If you want a system that instead of sending total counts sends increments, OpenTSDB doesn't need to calculate the sum, it just needs different aggregate function implementations and the inverse of "Rate". This would be much simpler and much more broadly applicable.

[tcurdt]

Wow, this is an old issue - and still open?

Not sure I can follow #184 or f53aad6

@manolama Is ZIM the non-interpolation aggregation one can use for counts as described by @tsuna?

[manolama]

@tcurdt Yes, the ZimSum supplies zero if missing so it avoids interpolation. Same with mimmax and mimmin.