chrissnell/crabby

A web page performance tester with support for Prometheus, InfluxDB, Graphite, Datadog, and Riemann

crabby's Issues

Allow setting TTL in Riemann storage

By default, Crabby will set TTL to 0, when Riemann storage is used. This won't work properly if scan interval is larger than TTL (which is), because events would be expired and removed from Riemann index.

See the following from http://riemann.io/howto.html:

If you're indexing events, but they don't appear in queries, they may have expired from the index.
Try (expired #(warn "expired" %)) to warn about expired events. They may be expiring because their
TTL is shorter than the interval between events.

To address this, Riemann storage should have a custom ttl key with something larger than 0. Usual (default) value is 5 or 10.

No internal metrics and TTFB in Riemann store

With this configuration:

general:
  request-timeout: 15s
  report-internal-metrics: true
  internal-metrics-gathering-interval: 10

jobs:
  - name: site
    type: simple
    url: "https://site"
    interval: 10

storage:
  riemann:
    host: localhost
    port: 5555
    metric-namespace: crabby

only these metrics are present:

crabby.site.dns_duration
crabby.site.server_response_duration
crabby.site.tls_handshake_duration

No internal and TTFB metrics. What about server load, is that part of selenium type?

Support loading of remote job lists

Currently, the user must configure all crabby jobs on the crabby server itself. This is not an ideal workflow for a business because it requires developers to work with the operations team to get their URLs set up. A better approach is to allow polling of configuration from a remote URL. Each developer could provide a URL that returns a YAML containing all of their crabby jobs, which would be loaded into crabby and run with the rest of the jobs.

The URL might look like this:

https://my.service/crabby.yaml

It could be a static file on the remote server or it could be dynamically generated by the app and could even include cookies that are automatically refreshed by the app so that crabby always has a valid authentication token.

The remote YAML would be re-polled on a configurable interval--probably something like once every five or ten minutes.

Support Riemann as a metrics output

Like it says. Optionally send all collected metrics to a remote Riemann endpoint.

Basic Auth

I didn't see anything in the docs or code on how to setup basic auth, do you know if this is supported?

Prometheus Pushgateway

It seems Prometheus is not working at all.

First, the prometheus does not accept . as label separator.
https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
Accepts regex [a-zA-Z_:][a-zA-Z0-9_:]*
Second issue seems to be "duplicate metrics collector registration attempted"

Prometheus pushgateway integration

Hi,

Is crabby's storage backend support prometheus pushgateway currently? I saw in CONFIGURATION.md mention that only support datadog and graphite. I'm not able to connect to prometheus in my playground setup as well.

New release?

Hello and thank you for this tool.
The latest release, v1.1, is from six years ago while the latest commit is only two-years old. There have been more than 100 commits since the latest release.
Is the current master branch stable? If not, could you tag the latest stable commit so we have a release that includes all/most of the latest changes?
Thanks in advance