Giter Site home page Giter Site logo

hanaasagi / tokio-metrics-collector Goto Github PK

View Code? Open in Web Editor NEW
21.0 3.0 3.0 43 KB

Provides utilities for collecting Prometheus-compatible metrics from Tokio runtime and tasks.

Home Page: https://crates.io/crates/tokio-metrics-collector

License: Apache License 2.0

Rust 100.00%
collector metrics prometheus tokio

tokio-metrics-collector's Introduction

Tokio metrics collector for Prometheus

tokio-metrics-collector on GitHub Action codecov tokio-metrics-collector on crates.io tokio-metrics-collector on docs.rs

Provides utilities for collecting Prometheus-compatible metrics from Tokio runtime and tasks.

[dependencies]
tokio-metrics-collector = { version = "0.2.1" }

Documentation:

QuickStart

use prometheus::Encoder;

#[tokio::main]
async fn main() {
    // register global runtime collector
    prometheus::default_registry()
        .register(Box::new(
            tokio_metrics_collector::default_runtime_collector(),
        ))
        .unwrap();

    // register global task collector
    let task_collector = tokio_metrics_collector::default_task_collector();
    prometheus::default_registry()
        .register(Box::new(task_collector))
        .unwrap();

    // construct a TaskMonitor
    let monitor = tokio_metrics_collector::TaskMonitor::new();
    // add this monitor to task collector with label 'simple_task'
    // NOTE: duplicate labels in multiple monitors cause incorrect data aggregation.
    // It is recommended to use unique labels for each monitor and
    // instrument multiple tasks by the `instrument` function.
    task_collector.add("simple_task", monitor.clone());

    // spawn a background task and instrument
    tokio::spawn(monitor.instrument(async {
        loop {
            // do something
            tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;
        }
    }));

    // print metrics every tick
    for _ in 0..5 {
        tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;

        let encoder = prometheus::TextEncoder::new();
        let mut buffer = Vec::new();
        encoder
            .encode(&prometheus::default_registry().gather(), &mut buffer)
            .unwrap();
        let data = String::from_utf8(buffer.clone()).unwrap();

        println!("{}", data);
    }
}

And a http server example, you can find in examples/server.rs.

Runtime Metrics

This unstable functionality requires tokio_unstable, and the rt crate feature. To enable tokio_unstable, the --cfg tokio_unstable must be passed to rustc when compiling. You can do this by setting the RUSTFLAGS environment variable before compiling your application; e.g.:

RUSTFLAGS="--cfg tokio_unstable" cargo build

Or, by creating the file .cargo/config.toml in the root directory of your crate. If you're using a workspace, put this file in the root directory of your workspace instead.

[build]
rustflags = ["--cfg", "tokio_unstable"]
rustdocflags = ["--cfg", "tokio_unstable"]
  • workers_count | type: Gauge
    The number of worker threads used by the runtime.
  • total_park_count | type: Counter
    The number of times worker threads parked.
  • total_noop_count | type: Counter
    The number of times worker threads unparked but performed no work before parking again.
  • total_steal_count | type: Counter
    The number of tasks worker threads stole from another worker thread.
  • total_steal_operations | type: Counter
    The number of times worker threads stole tasks from another worker thread.
  • num_remote_schedules | type: Counter
    The number of tasks scheduled from outside of the runtime.
  • total_local_schedule_count | type: Counter
    The number of tasks scheduled from worker threads.
  • total_overflow_count | type: Counter
    The number of times worker threads saturated their local queues.
  • total_polls_count | type: Counter
    The number of tasks that have been polled across all worker threads.
  • total_busy_duration | type: Counter
    The amount of time worker threads were busy.
  • injection_queue_depth | type: Gauge
    The number of tasks currently scheduled in the runtime's injection queue.
  • total_local_queue_depth | type: Gauge
    The total number of tasks currently scheduled in workers' local queues.
  • elapsed | type: Counter
    Total amount of time elapsed since observing runtime metrics.
  • budget_forced_yield_count | type: Counter
    The number of times that a task was forced to yield because it exhausted its budget.
  • io_driver_ready_count | type: Counter The number of ready events received from the I/O driver.

Task Metrics

License

tokio-metrics-collector is distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT, and COPYRIGHT for details.

tokio-metrics-collector's People

Contributors

dependabot[bot] avatar dizda avatar hanaasagi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

tokio-metrics-collector's Issues

example panics

Hi there,

The provided example starts well and shows the information on the metrics page http://127.0.0.1:8000/metrics, but then panics at some point:

2023-05-02T09:35:27.129597Z DEBUG hyper::proto::h1::conn: incoming body is empty
thread 'tokio-runtime-worker' panicked at 'new: 0.000110958 >= past: 0.000119583', src/task.rs:272:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2023-05-02T09:35:27.132073Z DEBUG hyper::proto::h1::io: parsed 12 headers
2023-05-02T09:35:27.132113Z DEBUG hyper::proto::h1::conn: incoming body is empty
thread 'tokio-runtime-worker' panicked at 'new: 0.000110958 >= past: 0.000119583', src/task.rs:272:9
2023-05-02T09:35:27.133559Z DEBUG hyper::proto::h1::io: parsed 12 headers
2023-05-02T09:35:27.133587Z DEBUG hyper::proto::h1::conn: incoming body is empty
thread 'tokio-runtime-worker' panicked at 'new: 0.000110958 >= past: 0.000119583', src/task.rs:272:9
2023-05-02T09:35:27.134570Z DEBUG hyper::proto::h1::io: parsed 12 headers
2023-05-02T09:35:27.134593Z DEBUG hyper::proto::h1::conn: incoming body is empty
thread 'tokio-runtime-worker' panicked at 'new: 0.000110958 >= past: 0.000119583', src/task.rs:272:9
2023-05-02T09:35:27.135544Z DEBUG hyper::proto::h1::io: parsed 12 headers
2023-05-02T09:35:27.135570Z DEBUG hyper::proto::h1::conn: incoming body is empty
thread 'tokio-runtime-worker' panicked at 'new: 0.000110958 >= past: 0.000119583', src/task.rs:272:9
2023-05-02T09:35:27.136488Z DEBUG hyper::proto::h1::io: parsed 12 headers
2023-05-02T09:35:27.136517Z DEBUG hyper::proto::h1::conn: incoming body is empty
thread 'tokio-runtime-worker' panicked at 'new: 0.000110958 >= past: 0.000119583', src/task.rs:272:9
2023-05-02T09:35:27.137841Z DEBUG hyper::proto::h1::io: parsed 12 headers
2023-05-02T09:35:27.137874Z DEBUG hyper::proto::h1::conn: incoming body is empty

This is in debug release.
It panics after sending many requests to /send_email

Also I've noticed that instrumented_count doesn't decrease for "get_user".

# HELP tokio_task_instrumented_count The number of tasks instrumented.
# TYPE tokio_task_instrumented_count gauge
tokio_task_instrumented_count{task="email"} 1
tokio_task_instrumented_count{task="get_user"} 24

Add a metrics-rs backend

It'd be really nice if this crate supported not only the prometheus crate, but could also emit the metrics via the generic metrics-rs crate ecosystem. This could be neatly feature-gated as an alternative, allowing for easier integration of tokio metrics into existing prometheus metrics endpoints

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.