beam-telemetry/telemetry_metrics_prometheus

Complete Telemetry.Metrics Reporter solution for Prometheus

telemetry_metrics_prometheus's Issues

Ability to register multiple metrics with same name

I have a use case where I need to capture different telemetry events as a single prometheus metric. Part of the key in the telemetry event would be converted to prometheus label. I expected the below approach to work, but the second one overwrites the first one because we use the metric name as the id in the attach call. Any idea how I can handle this case?

[
  last_value([:job, :pending, :total],
    event_name: [:queue, :foo, :pending],
    tags: [:queue],
    tag_values: fn _ -> %{queue: :foo} end
  ),
  last_value([:job, :pending, :total],
    event_name: [:queue, :bar, :pending],
    tags: [:queue],
    tag_values: fn _ -> %{queue: :bar} end
  )
]

Document running multiple instances such as in an umbrella application

Document the patterns for running multiple instances, such as in an umbrella app. Include the child spec as demonstrated in the test suite and a sample prometheus scrape configuration.

Change the handler_id to use event_name instead of metric_name

Hi @bryannaegele,

We have the following scenario, we want to publish metrics from Ecto, with the same metric_name even though the event_name is different.

[
 %{metric_name: [:ecto, :query], event_name: [:my_app, :repo, :query]},
 %{metric_name: [:ecto, :query], event_name: [:my_app, :another_repo, :query]},
 %{metric_name: [:ecto, :query], event_name: [:another_app, :repo, :query]}
]

That way we can build only one dashboard with the same query for each panel at Grafana with a selector for each repo, retrieving the repo from labels.

And I think we can achieve that, changing the event handler_id to use event_name instead of metric_name. What do you think?

Add HTTPS support

TLS support can be accomplished by passing the appropriate options down to plug_cowboy and surfacing the option to users.

Incompatible with telemetry_metrics 0.5.0

After the changes to telemetry_metrics in 0.5.0, this project needs to be updated.

Reset after scrape

Hey Bryan, I am working on implementing more observability in my team's systems, and I stumbled across your project.

As I understand it, there is no documented way to reset metrics.

Would you accept a PR for a configuration option to reset on scrape?

Proposal: split server from project

Thanks for this library. I'd like to preface by saying that I have read https://github.com/bryannaegele/telemetry_metrics_prometheus/blob/master/docs/rationale.md#why-include-a-server

I'd like to propose splitting this library out into 2 parts, one dealing with the conversion of telemetry events to a prometheus friendly format, the other dealing with the web server side of things.

I agree with the inclusion of a server in terms of getting up and running quickly, however I think there are some downsides to this approach that make using the library difficult for certain use cases:

• Using a different version of cowboy (or gun which depends on cowlib) causing dependency issues.
• Using a different authentication method such as an IP whitelist or HTTP basic auth.
• Environments where exposing a different port is not possible, such as Heroku.

This library already does a lot of heavy lifting in handling the telemetry events. I think it could be split in 2 ways.

Extraction of web:

1. Remove the Router and server code from telemetry_metrics_prometheus into a separate project (such as telemetry_metrics_prometheus_web or telemetry_metrics_prometheus_plug)
2. Depend on telemetry_metrics_prometheus from telemetry_metrics_prometheus_web
3. Using telemetry_metrics_prometheus_web behaves as it does today, running its own webserver.

Extraction of core:

1. Leave the Router and server code in telemetry_metrics_prometheus and extract everything else into telemetry_metrics_prometheus_core
2. Depend on telemetry_metrics_prometheus_core from telemetry_metrics_prometheus
3. Using telemetry_metrics_prometheus behaves as it does today, users with different needs can use telemetry_metrics_prometheus_core.

If you agree with this proposal then I am happy to help in whatever way I can.

Unable to start multiple instances in a single VM

From a Plug.Cowboy perspective, should it be possible to run multiple instances of the Telemetry.Metrics.Prometheus in the same VM as long as they're configured to use different ports?

Our umbrella app normally runs with individual components in separate containers in production but all together in a single Erlang VM for end-to-end testing and I'm getting failed to connect: eaddrinuse error messages when more than one instance of the exporter plug web server is turned on at once. Each component runs an instance of the exporter via a module that pulls in the exporter via a using macro and runs its own instance under a different registered name. Each using macro feeds the different exporter instances a different port on startup but something is still registering a conflict.

Am I missing something about what Plug/Cowboy considers a requirement for a unique address?

Include a reference table of supported metrics

I believe all reporters should include a table with the name of the Telemetry.Metric and what is their equivalent on the report terminology. Something like this:

Telemetry Metric	XYZ Metric
counter	...
last_value	...
sum	...
summary	...
histogram	...

If a metric is not supported, that's a good place to document it too.

WDYT?

Support for summary metric type

Is there a technical reason summary is unsupported? It is the default summary type generated in phoenix projects. This is something I would be interesting in working on, but I thought I would ask before I spent time looking at it.

Divergence from Prometheus Python Client

Hey all, I was looking into some work between elixir and python, and I saw that this package and the prometheus python client might be diverging in the implementations. This file/line in particular are an example I found: https://github.com/prometheus/client_python/blob/master/prometheus_client/metrics.py#L610

Link to repo that use TelemetryMetricsPrometheus

Greetings,

I have read Hex documentation for TelemetryMetricsPrometheus, Telemetry.Metrics, OpentelemetryPhoenix
It probably should be enough, but I fail to get content from localhost:9568/metrics. I get a 200 Response, but 0 content-length.

A link to any repo using TelemetryMetricsPrometheus (and Phoenix, if possible), would be much appreciated.

From my application.ex

  def start(_type, _args) do
    :opentelemetry_cowboy.setup()
    OpentelemetryPhoenix.setup(adapter: :cowboy2)
    OpentelemetryEcto.setup([:excdf, :repo])

    metrics =  [
    Telemetry.Metrics.last_value("vm.memory.total", unit: :byte),
    Telemetry.Metrics.counter("http.request.count", unit: :byte),
    Telemetry.Metrics.sum("http.request.payload_size", unit: :byte)
    ]

    children = [
      # Start the Telemetry supervisor
      ExcdfWeb.Telemetry,
      {Phoenix.PubSub, name: ExcdfWeb.PubSub},
      {TelemetryMetricsPrometheus, [metrics: metrics]},
      # Start the Endpoint (http/https)
      ExcdfWeb.Endpoint
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: ExcdfWeb.Supervisor]
    Supervisor.start_link(children, opts)
  end