expediadotcom/haystack-ui

haystack-ui's Issues

Spans view doesn't work with zipkin

We get a needle not found when clicking on the spans view for a zipkin trace.

[Feature Request]: Support for overall Latency and Success% for selected time ranges

As a developer I would like to have the ability to see overall values for latency (for tp50, tp95 and tp99) and success % for the entire population for the selected time range. This would help us understand service/operation performance over long term along with the trend. The single number for each percentile displayed represents the value for the entire population of requests in the selected time range

Run suite for running haystack-ui with Zipkin as traces backend

Adding a simple run suite/docker-compose to test out haystack-ui against zipkin backend. This will make it easy to test out haystack-ui with zipkin and avoiding code drifts.

Service Graph Cpu Issues

Vizceral is a cpu hog which leads to issues leaving haystack-ui open on another monitor.

zipkin store uses v1 url

https://github.com/ExpediaDotCom/haystack-ui#haystack-ui-as-drop-in-replacement-for-zipkin-ui suggests v1 endpoint. While this is ok actually the UI currently consumes v2 data, which is incidentally pretty similar to your data format I think https://zipkin.io/zipkin-api/#/default/get_traces

Including X-HAYSTACK-INFRASTRUCTURE-PROVIDER onto search bar

Including this in the search will allow my team to search for service failures by environment. If its in AWS or DC.

Span Detail View Improvements

When viewing a detailed view of a merged span, it's not clear what you're actually looking at. Currently, you see a view of merged tags, merged logs, and the (server) raw span. This should be improved so the user knows more clearly what they are viewing.

add screen shots

(giphy capture used for above)

how to run without nodejs?

good ui for zipkin, i want nest to my application, how to run without nodejs?

codegen error : Server not starting

node -v
v11.10.1

npm -v
6.7.0

npm run start:dev --verbose

Output

npm info it worked if it ends with ok
npm verb cli [ '/usr/local/bin/node',
npm verb cli   '/usr/local/bin/npm',
npm verb cli   'run',
npm verb cli   'start:dev',
npm verb cli   '--verbose' ]
npm info using [email protected]
npm info using [email protected]
npm verb run-script [ 'prestart:dev', 'start:dev', 'poststart:dev' ]
npm info lifecycle [email protected]~prestart:dev: [email protected]
npm info lifecycle [email protected]~start:dev: [email protected]

> [email protected] start:dev /Users/prabhakarv/Documents/GitHub/haystack-ui
> npm run clean && npm run codegen && npm run test && concurrently -r "npm run watch" "wait-on public/bundles/report.html && npm run start:devserver"

npm info it worked if it ends with ok
npm verb cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'clean' ]
npm info using [email protected]
npm info using [email protected]
npm verb run-script [ 'preclean', 'clean', 'postclean' ]
npm info lifecycle [email protected]~preclean: [email protected]
npm info lifecycle [email protected]~clean: [email protected]

> [email protected] clean /Users/prabhakarv/Documents/GitHub/haystack-ui
> rm -rf public/bundles/ && rm -rf static_codegen

npm verb lifecycle [email protected]~clean: unsafe-perm in lifecycle true
npm verb lifecycle [email protected]~clean: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/prabhakarv/Documents/GitHub/haystack-ui/node_modules/.bin:/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/prabhakarv/Documents/GitHub/haystack-ui/node_modules/.bin:/usr/local/bin:/Users/prabhakarv/.rvm/gems/ruby-2.6.3/bin:/Users/prabhakarv/.rvm/gems/ruby-2.6.3@global/bin:/Users/prabhakarv/.rvm/rubies/ruby-2.6.3/bin:/Users/prabhakarv/.nvm/versions/node/v8.0.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/prabhakarv/.rvm/bin:/Users/prabhakarv/go/bin
npm verb lifecycle [email protected]~clean: CWD: /Users/prabhakarv/Documents/GitHub/haystack-ui
npm info lifecycle [email protected]~postclean: [email protected]
npm verb exit [ 0, true ]
npm timing npm Completed in 130ms
npm info ok
npm info it worked if it ends with ok
npm verb cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'codegen' ]
npm info using [email protected]
npm info using [email protected]
npm verb run-script [ 'precodegen', 'codegen', 'postcodegen' ]
npm info lifecycle [email protected]~precodegen: [email protected]
npm info lifecycle [email protected]~codegen: [email protected]

> [email protected] codegen /Users/prabhakarv/Documents/GitHub/haystack-ui
> mkdir -p static_codegen && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/  ./haystack-idl/proto/span.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/traceReader.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/anomaly/anomalyReader.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/subscription/subscriptionManagement.proto

/Users/prabhakarv/Documents/GitHub/haystack-ui/node_modules/grpc-tools/bin/protoc.js:41
    throw error;
    ^

Error: spawn /Users/prabhakarv/Documents/GitHub/haystack-ui/node_modules/grpc-tools/bin/protoc ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:246:19)
    at onErrorNT (internal/child_process.js:427:16)
    at processTicksAndRejections (internal/process/next_tick.js:76:17)
    at process.runNextTicks [as _tickCallback] (internal/process/next_tick.js:51:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:804:11)
    at internal/main/run_main_module.js:21:11
npm verb lifecycle [email protected]~codegen: unsafe-perm in lifecycle true
npm verb lifecycle [email protected]~codegen: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/prabhakarv/Documents/GitHub/haystack-ui/node_modules/.bin:/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/prabhakarv/Documents/GitHub/haystack-ui/node_modules/.bin:/usr/local/bin:/Users/prabhakarv/.rvm/gems/ruby-2.6.3/bin:/Users/prabhakarv/.rvm/gems/ruby-2.6.3@global/bin:/Users/prabhakarv/.rvm/rubies/ruby-2.6.3/bin:/Users/prabhakarv/.nvm/versions/node/v8.0.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/prabhakarv/.rvm/bin:/Users/prabhakarv/go/bin
npm verb lifecycle [email protected]~codegen: CWD: /Users/prabhakarv/Documents/GitHub/haystack-ui
npm info lifecycle [email protected]~codegen: Failed to exec codegen script
npm verb stack Error: [email protected] codegen: `mkdir -p static_codegen && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/  ./haystack-idl/proto/span.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/traceReader.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/anomaly/anomalyReader.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/subscription/subscriptionManagement.proto`
npm verb stack Exit status 1
npm verb stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
npm verb stack     at EventEmitter.emit (events.js:197:13)
npm verb stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
npm verb stack     at ChildProcess.emit (events.js:197:13)
npm verb stack     at maybeClose (internal/child_process.js:984:16)
npm verb stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:265:5)
npm verb pkgid [email protected]
npm verb cwd /Users/prabhakarv/Documents/GitHub/haystack-ui
npm verb Darwin 18.5.0
npm verb argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "codegen"
npm verb node v11.10.1
npm verb npm  v6.7.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] codegen: `mkdir -p static_codegen && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/  ./haystack-idl/proto/span.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/traceReader.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/anomaly/anomalyReader.proto && grpc_tools_node_protoc --js_out=import_style=commonjs,binary:./static_codegen/ --grpc_out=./static_codegen --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` -I ./haystack-idl/proto/api/ -I ./haystack-idl/proto/ ./haystack-idl/proto/api/subscription/subscriptionManagement.proto`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] codegen script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm verb exit [ 1, true ]
npm timing npm Completed in 359ms

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/prabhakarv/.npm/_logs/2020-01-21T14_38_59_924Z-debug.log
npm verb lifecycle [email protected]~start:dev: unsafe-perm in lifecycle true
npm verb lifecycle [email protected]~start:dev: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/prabhakarv/Documents/GitHub/haystack-ui/node_modules/.bin:/usr/local/bin:/Users/prabhakarv/.rvm/gems/ruby-2.6.3/bin:/Users/prabhakarv/.rvm/gems/ruby-2.6.3@global/bin:/Users/prabhakarv/.rvm/rubies/ruby-2.6.3/bin:/Users/prabhakarv/.nvm/versions/node/v8.0.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/prabhakarv/.rvm/bin:/Users/prabhakarv/go/bin
npm verb lifecycle [email protected]~start:dev: CWD: /Users/prabhakarv/Documents/GitHub/haystack-ui
npm info lifecycle [email protected]~start:dev: Failed to exec start:dev script
npm verb stack Error: [email protected] start:dev: `npm run clean && npm run codegen && npm run test && concurrently -r "npm run watch" "wait-on public/bundles/report.html && npm run start:devserver"`
npm verb stack Exit status 1
npm verb stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
npm verb stack     at EventEmitter.emit (events.js:197:13)
npm verb stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
npm verb stack     at ChildProcess.emit (events.js:197:13)
npm verb stack     at maybeClose (internal/child_process.js:984:16)
npm verb stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:265:5)
npm verb pkgid [email protected]
npm verb cwd /Users/prabhakarv/Documents/GitHub/haystack-ui
npm verb Darwin 18.5.0
npm verb argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "start:dev" "--verbose"
npm verb node v11.10.1
npm verb npm  v6.7.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start:dev: `npm run clean && npm run codegen && npm run test && concurrently -r "npm run watch" "wait-on public/bundles/report.html && npm run start:devserver"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start:dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm verb exit [ 1, true ]
npm timing npm Completed in 1135ms

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/prabhakarv/.npm/_logs/2020-01-21T14_38_59_956Z-debug.log```

Time selector shows raw timestamp after selecting a custom timeframe

Steps to repro: select custom time frame in USB time selector, press apply and search, open time selector after page transition

please enhance to work with messaging traces

In testing zipkin we noticed that a messaging trace doesn't render.

Here is it in zipkin

The data is messaging2.json here https://github.com/openzipkin/zipkin/tree/master/zipkin-ui/testdata

In Haystack it doesn't look so hot :)

dead already

fun title in malaysian slang. The zipkin quickstart dies recently. Mind looking into it?

Docker container does not respect resource limits

Scenario
Deploy a docker container with the latest haystack ui tag and specify a cpu limit.

Expected
The Node process respects the resource limits and only sees the number of cpus allocated.

Actual
The Node process sees all the cpus of the host.

Notes
The problem affects the performance of the server by doing an unnecessary number of forks here:

haystack-ui/server/utils/server.js

Line 55 in ee08389

cluster.fork();

This is worse when running in a highly virtualized platform like Kubernetes where it's common to define cpu (and memory!) limits for each container.

I believe this is similar to this issue that affects Java containers.

If Node does not provide a solution a possible workaround would be to expose the number of cpus as a property and allow users to override it.

npm install failure

Running npm install in office comes following error, how to fix this:

haystack-ui git:(master) npm install

> [email protected] install /Users/liangwang/Documents/Project/haystack-ui/node_modules/canvas
> node-gyp rebuild

Package cairo was not found in the pkg-config search path.
Perhaps you should add the directory containing `cairo.pc'
to the PKG_CONFIG_PATH environment variable
No package 'cairo' found
gyp: Call to './util/has_lib.sh freetype' returned exit status 0 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/Users/liangwang/.nvm/versions/node/v8.1.2/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:336:16)
gyp ERR! stack     at emitTwo (events.js:125:13)
gyp ERR! stack     at ChildProcess.emit (events.js:213:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:197:12)
gyp ERR! System Darwin 16.7.0
gyp ERR! command "/Users/liangwang/.nvm/versions/node/v8.1.2/bin/node" "/Users/liangwang/.nvm/versions/node/v8.1.2/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/liangwang/Documents/Project/haystack-ui/node_modules/canvas
gyp ERR! node -v v8.1.2
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/liangwang/.npm/_logs/2017-11-06T22_26_15_105Z-debug.log

[Feature Request] Zoom in for trace view

When viewing a trace with a lot of spans, it would be good to be able to zoom into a specific period of time within the trace, filtering only the spans in that specific timeline.

Support for additional comparisons beyond equality in Universal Search

For at least one of the tags that we use, the value of the tag is a numerical value based on a sliding scale rather than a distinct value. It would be great for Universal Search to support the ability to search on all values greater than a given value, or less than a given value, or between two different values.

For example, our tag is something like quality-rating with a scale from -10 to 10, where > 0 is better, and < 0 is worse. Currently, if I wanted to get all the bad ones, I'd have to make 10 different searches for quality-rating = -1, quality-rating = -2, etc.

OperationName should be whitelisted without a serviceName

It should be possible to do a search on operationName without an existing serviceName in the query. Currently you get a "key must be whitelisted" error.

[Feature Request] Support negation for search

Often when I am doing a search and I have narrowed down to my serviceName I would like to see trends and traces for everything except one specific operationName. Is there anyway that search can support this feature?

serviceName=my-service operationName!=slowCall

Trends, Alerts, and Service Graph tabs disappear

Steps to reproduce:

1. Navigate to the traces of a service, like https://<haystackUrl>/search?time.preset=1h&serviceName=<serviceName>&tabId=traces
2. Click a bar in the timeline:
   

Expected result:
The Trends page opens for the correct time window, with all UI components rendered properly

Actual result:
The Trends page opens, but the tab options are gone (see below)

Support wildcards within search

We have a use case where we have a bunch of values for certain tags, and those tags are generally related.

For instance, we have a region tag with values like:
region-a-<uuid1>
region-a-<uuid2>
region-a-<uuid3>
region-b-<uuid4>
region-b-<uuid5>
region-b-<uuid6>

As it exists currently, our users have to keep in mind the UUIDs that they're looking for. It's not unreasonable, but it would make things easier if, seeking all the traces from region a, they could simply put region: region-a-*

Universal search bar css breaks when too many key/value pairs are submitted

Universal Search can not search for tags with periods in their names in 1.0.4

We recently tried moving to a more recent version of Haystack UI that heavily featured Universal Search, and discovered that there is an issue with searching for certain tags with periods in the name, such as http.method

Double tabId in service graph URL

Selecting a node on serviceGraph when inside a serviceName context will add an extra tabId to the URL

npm install error with nodejs 12.13.1

gyp WARN EACCES attempting to reinstall using temporary dev dir "/root/haystack-ui-1.1.8/node_modules/canvas/.node-gyp"
gyp WARN install got an error, rolling back install
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack Error: EACCES: permission denied, stat '/root/haystack-ui-1.1.8/node_modules/canvas/.node-gyp/12.13.1'
gyp ERR! System Linux 3.10.0-1062.7.1.el7.x86_64
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /root/haystack-ui-1.1.8/node_modules/canvas
gyp ERR! node -v v12.13.1
gyp ERR! node-gyp -v v5.0.5
gyp ERR! not ok
npm WARN [email protected] requires a peer of acorn@^6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of ajv@^6.9.1 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-12-06T02_23_16_494Z-debug.log

Change trends-connector to fetch data from MT through tags based query.

We need to change Haystack UI to fetch trends from MT through tags based query instead of metric key query to support ExpediaDotCom/haystack-trends#155 fixes.

Sample curl calls :

http://metrictank:6060/render?target=duration%3Binterval%3DFifteenMinute%3Bmtype%3Dgauge%3BoperationName%3Dc29tZS1zcGFu%3Bproduct%3Dhaystack%3BserviceName%3Dc29tZS1zZXJ2aWNl%3Bstat%3D%2A%3Bunit%3Dmircoseconds&from=-24h&until=now&format=json&maxDataPoints=840

Essentially the target changes to something like this :

For duration :
target=duration;interval=FifteenMinute;mtype=gauge;operationName=sample-operation;product=haystack;serviceName=sample-service;stat=max;unit=mircoseconds

For count :
target=success-span;interval=FifteenMinute;mtype=gauge;operationName=sample-operation;product=haystack;serviceName=sample-service;stat=max;unit=short

Few of the things to keep in mind for query generation:

1. The target value is url encoded.
2. serviceName and OperationName would be encoded with MetricPointNameEncoder as it is happening now before URL encoding.

Clicks on bubbles in Service Performance graph sometimes fail

Occasionally, clicking on certain (usually smaller) service bubbles in the Service Performance graph does not trigger a redirect. Repeated clicks (sometimes as many as five or six) are needed to actually get the desired effect of going to the service's page.

We're currently running the following versions:

• 1.0.3 of haystack-ui (we've seen this in 1.0.1 as well)
• 1.0.5 of haystack-traces (we've seen this in 1.0.3 as well)
• 1.0.2 of haystack-trends (we've seen this in 1.0.1 as well)