druid-io/druid-console

An updated coordinator console that provides more insight into the status of your Druid cluster. This is not a data visualization console for Druid.

druid-console

This repository contains a standalone web application (angularjs in src/client/druid.coffee). The web app can be built into the druid source, and will be served by druid's built-in jetty server.

Standalone console build instructions

Prerequisites: node, compass

Install node packages: npm install

Install bower packages: bower install

The druid repo and this repo should be siblings under the same parent directory. The build-into-druid bash script will compile the coffeescript and the sass into the build directory, and then copy them into the druid source at ../druid/server/src/main/resources/static. If you wish to build to a different location, edit the DEST in build-into-druid.

Built-in proxy server

This repo also includes a webserver that can proxy to various druid installations if they are present in zookeeper. To run the server, export the dns name of your zookeeper and the discovery path into environment variables, and then run run-server, like so:

export ZK_HOSTNAME="zookeeper.sweet.com"
export ZK_SERVICE_DISC_PATH="/path/to/discovery"
./run-server

druid-console's Issues

Druid-Console

How to install this druid console as stand alone software

New index types break indexing service pages

As per #8 src/client/factories/iUtils.coffee relies on a strict set of parsing rules to make the indexing service page render properly.

There should be a safe fallback in the case of an unknown indexing service task naming scheme.

See #3 also

Duplicate links to 'old console'

Links to 'old console' can appear twice if the indexing service is not there

Comes from apache/druid#1817

npm install fails

I wonder which node version should be used to build the project.

For example if I use node v0.10.40 (npm v1.4.28) which is specified in package.json, I get:

> [email protected] install ...node_modules/node-sass
> node scripts/install.js


...node_modules/node-sass/node_modules/request/node_modules/hawk/node_modules/boom/lib/index.js:5
const Hoek = require('hoek');
^^^^^
SyntaxError: Use of const in strict mode.

If I use node v9.2.0 (npm v5.5.1) I get:

> [email protected] install ...node_modules/watchify/node_modules/fsevents
> node-gyp rebuild

gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
  CXX(target) Release/obj.target/fse/fsevents.o
In file included from ../fsevents.cc:6:
../../nan/nan.h:339:13: error: no member named 'New' in 'v8::String'
    return  _NAN_ERROR(v8::Exception::Error, errmsg);
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MacOS 10.12.6

Refresh spinner can spin forever on 'loading'

From apache/druid#1817

http://screencast.com/t/Cxh3rltYf

The refresh spinner can spin forever on 'loading'.

I can't install the dependences correctly when npm install.

Bellow is the exception:

npm ERR! code ETARGET
npm ERR! notarget No compatible version found: chronology@^0.7.5
npm ERR! notarget Valid install targets:
npm ERR! notarget 0.0.1-security
npm ERR! notarget
npm ERR! notarget This is most likely not a problem with npm itself.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! notarget
npm ERR! notarget It was specified as a dependency of 'druid-console'
npm ERR! notarget

Indexing service requires strict regex

The indexing service reporting parts require strict regular expressions to parse the tasks that the indexing service is doing. As such, there is no way to handle a task that does not conform to the expected naming convention, and no way to handle a task that has a name that is not one of the very standard names.

This ask is to allow the console to better handle non-standard or not-as-commonly-used task names.

Can I set the "env" for the console?

We have a prod and a staging install of Druid. It would be nice when looking at the coordinator console to be able to tell which we're looking at. (The URL doesn't work because it seems to redirect to an IP-based URL.)

I see in the code that the big text "druid cluster" can include an "env" instead of the word cluster. But how do I configure my Druid coordinator to tell it the env name so that this works?

can't edit loadRule for a dataSource whose all segments are in unused state

Scaling section not visible

As of version 0.9.1.1 "Scaling Activity" section (#scaling) is not visible in Chrome v53 and Firefox v48.

The function getScaling (probalbly indexing.coffee#L61) fails as scaling parameter is an empty sring and map function is not defined.

curl -v http://<url>/druid/indexer/v1/scaling returns 200 without any content.
Autoscaling isn't enabled through config.

Are you aware of any potentioal cause?

some thoughts

Currently we was thinking about implementing interface for configuring druid indexing tasks. Firstly, i was trying to discover sources in main druid repo, but then i find out this repo.
and i have few questions :

1. why did this druid-console is not included in main repository?
2. did this version of console included in druid releases?
3. what do you think about to moving to some modern stack, like webpack+ts
4. what do you think about replacing angularjs( which is really old) with react or vue?
   i think ui of druid can have really big potential, for example add permission management and configuring kafka ingestion(flatten/transformation/columns,etc).
   But currently this repo looks some kind outdated. ts and react or vue is widely used this times, and it will be much easier for community to work on this ui.

also this repo contains very much stuff what can be done using IDE. for example here is watch task, which did running using bash script 😕(it is like running java application for debug using bash script) also such things make this repo platform dependent and working on ui under windows is not possible ( btw, druid is simple to debug/compile under windows, only tar task in distribution is not builded)

sql query by "not in" not work

I use the 0.15.0 version sql query,
e.q.
select a.uuid from db a where a.uuid not in (select b.uuid from db b where b.status = '0' )
it always return "LIMIT 5000 / org.apache.druid.java.util.common.ISE" but use "in" that ok.Am I using the error?

thanks~

404 when task type is index

Error when index id likes index_{datasource_name}_2018-03-20T17:39:09.499Z:
druid.js:138 TypeError: Cannot read property '3' of null
at Object.parseTaskId (druid.js:13739)
at Object.decorateTask (druid.js:13752)
at tasks.reduce.all (druid.js:13778)
at Array.reduce ()
at Object.processTasks (druid.js:13775)
at druid.js:13856
at druid.js:118
at J (druid.js:147)
at J (druid.js:147)
at druid.js:148

The root cause is regex for taskId parsing does not support index type task. Code is at

druid-console/src/client/factories/iUtils.coffee

Line 5 in 6f5137a

m = taskId.match /^((hadoop_convert_segment)|index_(hadoop|realtime|spark)|(archive))_(.+)_(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}Z)/