Comments (5)
Consul
Use consul for service discovery. The Prometheus exporter will be extended to optionally support Consul.
To use Consul support, add the consul
section to the Prometheus exporter. The consul section allows pollers to export to Prometheus without having to specify the Prometheus address and port for each poller individually. Exporters that include the consul
block create an HTTP endpoint to serve the ONTAP metrics data, just like the standard Prometheus exporter, with three important differences.
- A random free port will be picked for the open-metrics endpoint
- Harvest will register that endpoint with Consul
- Harvest will monitor and update Consul on the health of the poller
We will tell Prometheus to ask Consul for the list of targets to scrape by using Prometheus' consul_sd_configs
support. Let's take a look at an example.
Add an exporter to harvest.yml
like so:
Exporters:
pluto: # name of your exporter, can be any valid yaml string
exporter: Prometheus # type of exporter - Prometheus in this case
consul: # enable service discovery via Consul
host: 1.2.3.4 # ip or hostname of Consul cluster
port: 8500 # port of Consul cluster, defaults to Consul's default 8500
See consul configuration options for the full list of configuration options.
Configure Prometheus to find targets from Consul
Continuing with the example, we tell Prometheus to scrape our Consul registered pollers by adding a new scrape config to the prometheus.yml
and using the consul_sd_configs
section like so:
scrape_configs:
- job_name: 'harvest'
scrape_interval: 60s
consul_sd_configs:
- server: 'localhost:8500'
services:
- 'harvest-poller' # this name matches the service name defined in harvest.yml
This tells Prometheus to ask Consul (running on localhost:8500
) for a list of targets associated with the harvest-poller
service. The service name matches the service named defined in harvest.yml
, which defaults to harvest-poller
.
See Prometheus Consul_sd_config for the full list of ways to configure consul_sd_configs
.
Consul Config Options
NOTE When consul
is enabled, the port
defined in the Prometheus
section will be ignored. That's because service discovery will assign the port. You can omit the port
parameter altogether when using consul
.
An overview of Consul parameters defined as children of the consul
block:
parameter | type | description | default |
---|---|---|---|
host |
string, required | ip or hostname of Consul cluster | |
port |
int, optional | port of Consul cluster | 8500 |
service_name |
string, optional | service name, should be valid DNS label | harvest-poller |
ttl |
string, Go duration | Consul TTL check - amount of time before Consul considers poller down | 5m (five minutes) |
tags |
list of strings, optional | list of tags to assign to this service |
Example:
Exporters:
pluto: # name of your exporter, can be any valid yaml string
exporter: Prometheus
port: 12990 # ignored since this exporter contains a consul block
consul: # consul block added because we want service discovery
host: 1.2.3.4 # ip or hostname of Consul cluster
service_name: monitor # all pollers exporting to pluto will be registered with the service name of 'monitor'
tags:
- netapp # each of the pollers exporting to pluto will be tagged with two tags: 'netapp' and 'harvest'
- harvest
References
from harvest.
Comment from Vachagan:
- file-based SD would be very easy to implement, unfortunately it only works if Prometheus runs on the same machine as Harvest (usually not the case)
- HTTP-based SD is discussed by the Prometheus devs, but not clear if it will be implemented anytime soon.
- Using the Adapter mentioned in your second link may be the most realistic - needs more investigation
from harvest.
this looks great, thanks for the detailed breakdown.
the only downside is that users have to install an additional software where Prometheus is running.
from harvest.
Closed by #575
from harvest.
Hi Cris,,
Prometheus services stopped after few mins of start up. i see starting up replaying WAL prompt and services stopped after some time. May i help get fix this issue? Thanks
from harvest.
Related Issues (20)
- how to compute metric volume_size from GB to TB HOT 1
- Harvest should remove Workload dashboard > Service Center row HOT 1
- Harvest should allow exporting metrics without a prefix HOT 1
- Invalid syntax in 24.05 when VolumeAnalytics enabled HOT 11
- Harvest should handle HTTP 414 responses from REST endpoints
- Harvest should collect virtual_used and virtual_used_percent metrics for REST volume template HOT 1
- vscan plugin should handle IPv6 scanners HOT 2
- Harvest should collect quotas even when there are no qtrees HOT 2
- Throughout metric in node dashboard should use sum instead of avg as aggregation HOT 1
- Feature request for Dashboard template for ontaps3 metrics HOT 1
- fetching volume timeout HOT 4
- Qtree duplicate keys seen in Zapi plugin when quotas are higher than batching size with user/group quotas enabled HOT 1
- Node Power Metrics not pulled from 9.15.1RC1 or A90/A1K HOT 1
- Harvest should remove historicalLabels from REST qtree template HOT 1
- Quota dashboard should use kibibytes instead of kilobytes HOT 1
- Harvest should provide a FIPS compliant image HOT 1
- Release 24.08.0
- Tenants Dashboard should include column: average size per object HOT 2
- Buckets board should also include "Avg Size per Object"
- Harvest should improve `bin/harvest generate metrics` tool to include metrics created via plugins
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from harvest.