brimdata/grafana-zed-datasource

Grafana data source plugin for Zed lakes

grafana-zed-datasource's Issues

Code editor

In a Grafana community Slack thread, someone pointed at https://github.com/grafana/grafana/blob/901495a54ae36ecafb2ef5bd45ac07b2c8f03bb4/packages/grafana-ui/src/components/Monaco/CodeEditor.mdx as a code editor. Maybe look at using this for the Zed query instead of TextArea.

Query variables

As the README explains, in its first version the plugin only supports custom variables. These docs describe how to add support for variables whose values are pulled from the data source, which should be made possible.

Rewrite as a backend plugin (needed for auth & Query Inspector)

The plugin was recently rewritten to use the zed-js client to query the Zed lake, since this brings improvements in code efficiency and readability. However, having completed the effort, a couple shortcomings are now apparent.

1. The Query tab of Grafana's Query Inspector no longer works with this plugin. If the Refresh button is clicked, it just spins. grafana/31459 explains why this is expected.

   Grafana cannot detect that that URL is going against a datasource query endpoint, only those that query our backend data source api's or data source proxy api will show up in the query inspector

2. If we wanted to add authentication to the plugin (#8), Grafana's docs explain how being a "backend plugin" would be a prerequisite for a clean implementation of this.

For now the plugin isn't missing out on much by being written the way it is. The info that would normally be in Query Inspector can instead be found in browser tools, e.g., the Network tab of the Chrome DevTools console. And while the Zed lake has some limited authentication support, it doesn't yet support a flavor that would be compatible with Grafana's plugins. So we can cross this bridge when user demand justifies it.

@jameskerr offered a high-level sketch of one way we might go about doing the rewrite. What we could potentially do is go back to having the plugin hit the Zed lake API "raw" but as a backend plugin that requests query responses in ZJSON just as zed-js does, then use zed-js to decode the ZJSON.

Handle string-typed Time Fields

A community zync user attempted to use the plugin and happened to have a string-typed field that held a timestamp value and was stumped when the plugin wasn't able to plot data just by specifying it as the Time Field. While won't be as efficient to do the conversions on the fly at the JavaScript layer, it would surely be user friendly if the plugin could somehow handle this case automatically (and perhaps issue a warning to the user so they know they're not seeing ideal performance.)

For now, a change in #14 has made it such that the plugin does detect when the user has picked a non-time field and surfaces an error message advising them as to why it's not working.

Authentication support

Right now the plugin only supports access to an unauthenticated Zed service such as a zed serve started with defaults or the one that runs on localhost behind Zui. However, a Zed service may be configured to require authentication, so the plugin should be enhanced to work with this. Preliminary research has indicated that the approaches for auth currently available in Zed would not be workable with the plugin, so additional work will be required on the Zed side.

Quote the pool name in "from"

I recently used the plugin to try to access a pool that had a hyphen in it, which caused a syntax error. For now one can just wrap the pool name in quotes right in the Query Editor, but the plugin should instead handle this for the user.

Use zed-js client

In its first version, the plugin accesses the Zed API directly over HTTP and receives a query result as JSON. This requires it to issue some bonus mini queries to do things like check the shape count or grab the field names & types for assembling the data frame. However, the zed-js client that @jameskerr is working on would allow it to issue a single query and get back all that meta-info as a side effect, which would allow the removal of a significant amount of code. @jameskerr also suggested we might move the plugin into the planned Zealot "monorepo" so it can live alongside its zed-js dependency. I can look at doing both of these things at the same time when Zealot is ready.