Utility to extract metrics from arbitary data stores in Prometheus format
Export your custom app metrics from external data stores like PostgreSQL, MySQL, Redis(coming soon!)
- Extract column names from results and expose them as custom metric labels.
- Ability to register multiple jobs with different stores.
store-exporter loads SQL query file and fetches the data from DB and transforms the result in Prometheus text format. A lot of times, it is undesirable to add instrumentation right in your app for the following reasons:
- Your app doesn't have any HTTP server, but to just extract metrics you've to invoke HTTP server.
- Your app cares about being fast in which case adding any external library penalises performance.
- You don't want to mix the app logic with the metric collection/exposition logic.
In all the above cases, it is more suitable to take a Sidecar approach, where you query for metrics from an external persistent store your app maintains. This utility just makes it easier for anyone to write custom SQL queries and expose metrics without having to worry about Prometheus format/exposition logic. You can run a single binary anywhere in your cluster environment which has access to the external store which exposes the metrics on an HTTP server confirming to Prometheus metric format.
There are multiple ways of installing store-exporter.
docker run -p 9609:9609 -v /etc/store-exporter/config.toml:/etc/store-exporter/config.toml mrkaran/store-exporter:latest
Precompiled binaries for released versions are available in the Releases section.
You can checkout the source code and build manually:
git clone https://github.com/mr-karan/store-exporter.git
cd store-exporter
make build
cp config.sample.toml config.toml
./store-exportermkdir store-exporter && cd store-exporter/ # copy the binary and config.sample in this folder
cp config.toml.sample config.toml # change the settings like server address, job metadata, db credentials etc.
./store-exporter # this command starts a web server and is ready to collect metrics.You can send a GET request to /metrics and see the following metrics in Prometheus format:
# HELP job_name_basicname this is such a great help text
# TYPE job_name_basicname gauge
job_name_basicname{job="myjob",pg_db_blks_hit="74400",pg_db_tup_inserted="120"} 13713
# HELP job_name_verybasic_name this is such a great help text again
# TYPE job_name_verybasic_name gauge
job_name_verybasic_name{job="myjob",pg_db_conflicts="0",pg_db_temp_bytes="0"} 40
# HELP version Version of store-exporter
# TYPE version gauge
version{build="846771f (2019-08-28 10:28:07 +0530)"} 1-
[server]
- address: Port which the server listens to. Default is 9608
- name: Optional, human identifier for the server.
- read_timeout: Duration (in milliseconds) for the request body to be fully read) Read this blog for more info.
- write_timeout: Duration (in milliseconds) for the response body to be written.
-
[app]
- log_level: "production" for all
INFOlevel logs. If you want to enable verbose logging use "debug". - jobs
- name: Unique identifier for the job.
-- store: Config options for the store
- db: Type of SQL DB. Supported values: [postgres, mysql].
- dsn: Connection URL to the DB.
- query: Path to SQL file.
- max_open_connections: Max open connections to the DB.
- max_idle_connections: Max idle connections maintained in the connection pool.
- metrics:
- namespace: Unique identifier for the metric, prepended in each metric name.
- help: Helptext for the metric
- query: Name of the query mapped in
sqlfile, used to query the db for this metric. - labels: List of additional column names fetched from the DB, to be used in metric as key/value pairs.
- columns: Each column name creates a separate Prometheus
metricand the corresponding value fetched from the store is used as the metric value.
- name: Unique identifier for the job.
-- store: Config options for the store
- log_level: "production" for all
NOTE: You can use --config flag to supply a custom config file path while running store-exporter.
You can add the following config under scrape_configs in Prometheus' configuration.
- job_name: 'store-exporter'
metrics_path: '/metrics'
static_configs:
- targets: ['localhost:9610']
labels:
service: my-app-metricsValidate your setup by querying version to check if store-exporter is discovered by Prometheus:
`version{build="846771f (2019-08-28 10:28:07 +0530)"} 1`
PRs on Feature Requests, Bug fixes are welcome. Feel free to open an issue and have a discussion first. Contributions on more external stores are also welcome and encouraged.
Read CONTRIBUTING.md for more details.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent