Separate data sink from web interface

[opentsdb]

Separate data sink from web interface

[tsuna]

Why?

[anomalizer]

I see TSDB daemon doing the following:

1. Accept data write options from tcollector and other such apps over telnet style RPC
2. Allow data retrieval over HTTP (as ascii, charts, and also stats)
3. A web interface for humans to query

From a deployment perspective, I'd like (1) & (2+3) to be separate as a way to do some access-control i.e. a deployment of (2+3) acts as a read-only system.

From a compilation and packaging perspective, my guess is that only (3) needs GWT and I'd like to keep that as separate unit (given its size contribution to the final build). Also, I am not a big fan of the UI, so I'd like to have (2) without getting (3). I was hoping to ask for this split as a request at a later point but the cat is now out of the bag.

[tsuna]

We could add a flag to selectively disable (1) or (2). Nothing requires (3), so if you disable (2) you can just ignore (3) altogether.

[manolama]

I'm working on this now where you can configure the TSD to perform one of different rolls, one of them will be just an ingest for data, another will be a GUI, etc.

[choang]

I'm just looking into opentsdb for the past week, and found (2) to be a bit frustrating. I don't plan on using the web UI, so I wanted to test the HTTP API, but I could only get ascii to work. The json query requires gnuplot, so it seems like a bit of an overhead for querying the system. I experienced this with both the master and next branch builds.

I'm not a Java dev, so going through the code on the next branch, I don't see the decoupling of the UI elements from the API. Is that on the roadmap?

My plan is to use jQuery charts so I'm just interested in a JSON response, and not interested in the PNG charts at all.

[manolama]

Take a look at the latest commits in the next branch. It implements the /api/query endpoint that lets you retrieve data as JSON. You still need GNUPlot to compile, for now, but in the future that may be moved to a plugin. Docs are at http://opentsdb.net/docs/build/html/api_http/query.html

[choang]

This is exactly what I was looking for. Thanks! (And sorry I didn't find it myself.)

[choang]

Is there a reason you chose to create the datapoints as a dictionary instead of list? Since they are stored in time order (I read that somewhere), I think returning a list would be better, so I don't have to sort them when I try to plot.

[manolama]

They are actually returned in order, it's up to whatever parses them to determine if they'll be unsorted. But you can also get them as a list with the &arrays query string parameter. Also, please post to the mailing list or another item since the API isn't directly related to this issue. Thanks!

[johann8384]

I don't think this request is needed anymore. Although I may be proposing a similar change to the mailing list to support SSL or SSH for better security and authentication support.