prometheus-community / elasticsearch_exporter Goto Github PK
View Code? Open in Web Editor NEWElasticsearch stats exporter for Prometheus
License: Apache License 2.0
elasticsearch_exporter's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Stargazers
Watchers
Forkers
dan-cleinmark mattkanwisher nadnerb keis wehkamp tamsky lowstz neowaylabs deedubs bblok11 jeinwag expressenab nodestory gitter-badger trotttrotttrott amorken dylanmei flecno erezny prfalken william-yeh overtimedog jbdalido binman-docker echocat jrummler closeio deepthawtz betterdoctor nvx finn-no rafael-0317 richerve edschouten bobrik tanner-bruce genpage jnials nyodas mkocikowski dominikschulz htchang record nuagen xiu timonwong objectrocket wyukawa rajughoshdev quantumghost traum-ferienwohnungen etiennecoutaud decsis leonidas ghellings f0 robothor lesterwang jesseolsen kuckkuck yackushevas psiphon-inc dannyk81 sapcc jmcarp gfanton neumayer zwopir algestam thedodd georgegoh jsheflin fatelei kylemcc rayhou caitong93 matsumana jwhitcraft yangchangshun metavige yanns ywww ktarow pcittadini ildarmf torresdal haizaar abcddpro mehrdad-shokri lloydgriffiths tillkolter gjaminon fcgravalos pchristos coderchenj carlosmrbcn difrex ppizarro bonniernews dogopupperelasticsearch_exporter's Issues
Typo in readme
| elasticsearch_indices_indexing_index_time_seconds_total | counter | 1 | Total index calls |
|---|---|---|---|
| elasticsearch_indices_indexing_index_total | counter | 1 | Cumulative index time in seconds |
I believe it should be vice-versa
Docker image?
Would be lovely to have a docker image for this.
I tried building with a Dockerfile that matches the consul_exporter, but I get the following error:
# Executing 4 build triggers
Trigger 0, COPY . /app
Step 0 : COPY . /app
Trigger 1, RUN /bin/go-build
Step 0 : RUN /bin/go-build
---> Running in 9eca28664377
fetch http://dl-4.alpinelinux.org/alpine/v3.2/main/x86_64/APKINDEX.tar.gz
(1/25) Installing run-parts (4.4-r0)
(2/25) Installing openssl (1.0.2d-r0)
(3/25) Installing lua5.2-libs (5.2.4-r0)
(4/25) Installing lua5.2 (5.2.4-r0)
(5/25) Installing ncurses-terminfo-base (5.9-r3)
(6/25) Installing ncurses-widec-libs (5.9-r3)
(7/25) Installing lua5.2-posix (33.3.1-r2)
(8/25) Installing ca-certificates (20141019-r2)
(9/25) Installing libssh2 (1.5.0-r0)
(10/25) Installing curl (7.42.1-r0)
(11/25) Installing expat (2.1.0-r1)
(12/25) Installing pcre (8.37-r1)
(13/25) Installing git (2.4.1-r0)
(14/25) Installing libbz2 (1.0.6-r3)
(15/25) Installing libffi (3.2.1-r0)
(16/25) Installing gdbm (1.11-r0)
(17/25) Installing ncurses-libs (5.9-r3)
(18/25) Installing readline (6.3.008-r0)
(19/25) Installing sqlite-libs (3.8.10.2-r0)
(20/25) Installing python (2.7.10-r1)
(21/25) Installing mercurial (3.4-r0)
(22/25) Installing bzr (2.6.0-r1)
(23/25) Installing make (4.1-r0)
(24/25) Installing vim (7.4.712-r0)
(25/25) Installing build-deps (0)
Executing busybox-1.23.2-r0.trigger
Executing ca-certificates-20141019-r2.trigger
OK: 131 MiB in 43 packages
(1/1) Installing tar (1.28-r0)
Executing busybox-1.23.2-r0.trigger
OK: 132 MiB in 44 packages
make: Warning: File 'Makefile.COMMON' has modification time 73 s in the future
make: warning: Clock skew detected. Your build may be incomplete.
make: Warning: File 'Makefile.COMMON' has modification time 73 s in the future
make: warning: Clock skew detected. Your build may be incomplete.
make: Warning: File 'Makefile.COMMON' has modification time 73 s in the future
Go version 1.4.2 required but not found in PATH.
About to download and install go1.4.2 to /app/.build/go1.4.2
Abort now if you want to manually install it system-wide instead.
mkdir -p /app/.build/go1.4.2
curl -L https://golang.org/dl/go1.4.2.linux-amd64.tar.gz | tar -C /app/.build/go1.4.2 --strip 1 -xz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 87 100 87 0 0 201 0 --:--:-- --:--:-- --:--:-- 235
100 59.5M 100 59.5M 0 0 640k 0 0:01:35 0:01:35 --:--:-- 728k
mkdir -p /app/.build/gopath/src/github.com/prometheus/
ln -s /app /app/.build/gopath/src/github.com/prometheus/elasticsearch_exporter
GOROOT=/app/.build/go1.4.2 GOPATH=/app/.build/gopath /app/.build/go1.4.2/bin/go build -o elasticsearch_exporter
elasticsearch_exporter.go:15:2: cannot find package "github.com/prometheus/client_golang/prometheus" in any of:
/app/.build/go1.4.2/src/github.com/prometheus/client_golang/prometheus (from $GOROOT)
/app/.build/gopath/src/github.com/prometheus/client_golang/prometheus (from $GOPATH)
Makefile.COMMON:95: recipe for target 'elasticsearch_exporter' failed
make: *** [elasticsearch_exporter] Error 1
The command '/bin/sh -c /bin/go-build' returned a non-zero code: 2
Error while querying Elasticsearch for nodes stats
Hi,
i run the latest version (0.3.3) with kubernetes in the same pod as a elasticsearch client node
I see this error in te logs, and in the /metrics , only the exporter internal metrics are present
kubectl logs es-client-286494296-74hnb es-exporter -n logging -f
2017/06/16 15:06:58 Starting Server: :9108
2017/06/16 15:07:25 Error while querying Elasticsearch for nodes stats: Get http://127.0.0.1:9200/_nodes/_local/stats: dial tcp 127.0.0.1:9200: getsockopt: connection refused
if i run the same via curl .... in the same container ... it does work, any ideas?
kubectl exec -ti es-client-286494296-74hnb -c es-exporter curl http://127.0.0.1:9200/_nodes/_local/stats -n logging
{"_nodes":{"total":1,"successful":1,"failed":0},"cluster_name":"1.kube-logging","nodes":{"lOvgYpyoT9SZ7tNHJhuihg":{"timestamp":1497619887846,"name":"es-client-286494296-74hnb","transport_address":"10.2.1.140:9300","host":"10.2.1.140","ip":"10.2.1.140:9300","roles":["ingest"],"indices":{"docs":{"count":0,"deleted":0},"store":{"size_in_bytes":0,"throttle_time_in_millis":0},"indexing":{"index_total":0,"index_time_in_millis":0,"index_current":0,"index_failed":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0,"noop_update_total":0,"is_throttled":false,"throttle_time_in_millis":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0,"scroll_total":0,"scroll_time_in_millis":0,"scroll_current":0,"suggest_total":0,"suggest_time_in_millis":0,"suggest_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0,"total_stopped_time_in_millis":0,"total_throttled_time_in_millis":0,"total_auto_throttle_in_bytes":0},"refresh":{"total":0,"total_time_in_millis":0,"listeners":0},"flush":{"total":0,"total_time_in_millis":0},"warmer":{"current":0,"total":0,"total_time_in_millis":0},"query_cache":{"memory_size_in_bytes":0,"total_count":0,"hit_count":0,"miss_count":0,"cache_size":0,"cache_count":0,"evictions":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"completion":{"size_in_bytes":0},"segments":{"count":0,"memory_in_bytes":0,"terms_memory_in_bytes":0,"stored_fields_memory_in_bytes":0,"term_vectors_memory_in_bytes":0,"norms_memory_in_bytes":0,"points_memory_in_bytes":0,"doc_values_memory_in_bytes":0,"index_writer_memory_in_bytes":0,"version_map_memory_in_bytes":0,"fixed_bit_set_memory_in_bytes":0,"max_unsafe_auto_id_timestamp":-9223372036854775808,"file_sizes":{}},"translog":{"operations":0,"size_in_bytes":0},"request_cache":{"memory_size_in_bytes":0,"evictions":0,"hit_count":0,"miss_count":0},"recovery":{"current_as_source":0,"current_as_target":0,"throttle_time_in_millis":0}},"os":{"timestamp":1497619887847,"cpu":{"percent":8,"load_average":{"1m":2.65,"5m":2.29,"15m":2.25}},"mem":{"total_in_bytes":135177543680,"free_in_bytes":104814276608,"used_in_bytes":30363267072,"free_percent":78,"used_percent":22},"swap":{"total_in_bytes":0,"free_in_bytes":0,"used_in_bytes":0}},"process":{"timestamp":1497619887848,"open_file_descriptors":805,"max_file_descriptors":1048576,"cpu":{"percent":0,"total_in_millis":25700},"mem":{"total_virtual_in_bytes":21566730240}},"jvm":{"timestamp":1497619887848,"uptime_in_millis":1409494,"mem":{"heap_used_in_bytes":867568984,"heap_used_percent":20,"heap_committed_in_bytes":4151836672,"heap_max_in_bytes":4151836672,"non_heap_used_in_bytes":69666280,"non_heap_committed_in_bytes":74113024,"pools":{"young":{"used_in_bytes":128411744,"max_in_bytes":1145372672,"peak_used_in_bytes":1145372672,"peak_max_in_bytes":1145372672},"survivor":{"used_in_bytes":142588752,"max_in_bytes":143130624,"peak_used_in_bytes":142588752,"peak_max_in_bytes":143130624},"old":{"used_in_bytes":596568488,"max_in_bytes":2863333376,"peak_used_in_bytes":596568488,"peak_max_in_bytes":2863333376}}},"threads":{"count":170,"peak_count":170},"gc":{"collectors":{"young":{"collection_count":2,"collection_time_in_millis":1030},"old":{"collection_count":1,"collection_time_in_millis":789}}},"buffer_pools":{"direct":{"count":111,"used_in_bytes":692142088,"total_capacity_in_bytes":692142087},"mapped":{"count":0,"used_in_bytes":0,"total_capacity_in_bytes":0}},"classes":{"current_loaded_count":9576,"total_loaded_count":9576,"total_unloaded_count":0}},"thread_pool":{"bulk":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"fetch_shard_started":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"fetch_shard_store":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"flush":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"force_merge":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"generic":{"threads":4,"queue":0,"active":0,"rejected":0,"largest":4,"completed":1549},"get":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"index":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"listener":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"management":{"threads":5,"queue":0,"active":1,"rejected":0,"largest":5,"completed":544},"refresh":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"search":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"snapshot":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0},"warmer":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0}},"fs":{"timestamp":1497619887849,"total":{"total_in_bytes":966052671488,"free_in_bytes":931348103168,"available_in_bytes":891423580160,"spins":"true"},"data":[{"path":"/data/data/nodes/0","mount":"/data (/dev/sda9)","type":"ext4","total_in_bytes":966052671488,"free_in_bytes":931348103168,"available_in_bytes":891423580160,"spins":"true"}],"io_stats":{"devices":[{"device_name":"sda9","operations":45293,"read_operations":150,"write_operations":45143,"read_kilobytes":8692,"write_kilobytes":4391960}],"total":{"operations":45293,"read_operations":150,"write_operations":45143,"read_kilobytes":8692,"write_kilobytes":4391960}}},"transport":{"server_open":67,"rx_count":3383,"rx_size_in_bytes":1043782,"tx_count":3383,"tx_size_in_bytes":5931618},"http":{"current_open":1,"total_opened":25},"breakers":{"request":{"limit_size_in_bytes":2491102003,"limit_size":"2.3gb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.0,"tripped":0},"fielddata":{"limit_size_in_bytes":2491102003,"limit_size":"2.3gb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.03,"tripped":0},"in_flight_requests":{"limit_size_in_bytes":4151836672,"limit_size":"3.8gb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.0,"tripped":0},"parent":{"limit_size_in_bytes":2906285670,"limit_size":"2.7gb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.0,"tripped":0}},"script":{"compilations":0,"cache_evictions":0},"discovery":{"cluster_state_queue":{"total":0,"pending":0,"committed":0}},"ingest":{"total":{"count":0,"time_in_millis":0,"current":0,"failed":0},"pipelines":{}}}}}
My ElasticSearchโs version is 2.2.1๏ผThe Grafana can not show Memory
My ElasticSearchโs version is 2.2.1๏ผThe Grafana can not show Memory
I want to know your esโs version
Document custom node exporter labels in grafana example
It looks like the sample dashboard uses metrics from the prometheus node exporter, like node_memory_MemTotal, that include the cluster label: https://github.com/justwatchcom/elasticsearch_exporter/blob/master/examples/grafana/dashboard.json#L156. This label isn't included in the node exporter by default, so I'm guessing you're adding custom labels in your scrape_configs. It would be useful to add a note describing this requirement and showing examples of how to add the custom labels, since several panels in the example dashboard won't work otherwise.
Document default values.
The default values aren't documented
For example.. what's the default for --es.all ? true or false?
cluster health not mentioned
no information about cluster healths, and what are the available, initialize then unassigned
Add support for --version command line parameter
So version can be determined in a manner similar to the official prometheus clients.
It should probably be formatted similar to the official clients also:
- prometheus, version 1.7.2 (branch: HEAD, revision: 22eadbe635528fa17b99a7635fed6b6018103042)
failed to decode cluster health
I am getting these warnings from the exporter when running against an elasticsearch cluster at version 5.4.1 with the prebuilt exporter 1.0.0-rc1.linux-amd64:
level=warn msg="failed to decode cluster health" err="json: cannot unmarshal number into Go struct field clusterHealthResponse.status of type string"
All of my configuration appears to be correct for the exporter. The elasticsearch cluster itself is healthy and in active use. I can query for the /_cluster/health manually using curl, and it returns the following:
{
"active_primary_shards": 141,
"active_shards": 348,
"active_shards_percent_as_number": 100.0,
"cluster_name": "<cluster_name>",
"delayed_unassigned_shards": 0,
"initializing_shards": 0,
"number_of_data_nodes": 3,
"number_of_in_flight_fetch": 0,
"number_of_nodes": 4,
"number_of_pending_tasks": 0,
"relocating_shards": 0,
"status": "green",
"task_max_waiting_in_queue_millis": 0,
"timed_out": false,
"unassigned_shards": 0
}
Any ideas as to what the issue may be?
how to distinguish two nodes on the same host
hi
if two nodes are on the same host, the metrics exposed both use the ip as the node name.
eg:
elasticsearch_thread_pool_active_count{cluster="ops-es",node="172.18.20.57",type="bulk"} 0
so how to distinguish metrics between two nodes?
Inconsistency in README
In the README file you refer to tag version 1.0.1 but some of the configuration arguments like es.indices and version are not available for that version tag.
First thing that you would do in this case is use the latest tag assuming there is a new release but that tag points to 1.0.1 instead of 1.0.2rc2 which would be latest.
It's a bit confusing.
flag provided but not defined: -es.indices
Usage of elasticsearch_exporter:
-es.all
Export stats for all nodes in the cluster.
-es.ca string
Path to PEM file that conains trusted CAs for the Elasticsearch connection.
-es.client-cert string
Path to PEM file that conains the corresponding cert for the private key to connect to Elasticsearch.
-es.client-private-key string
Path to PEM file that conains the private key for client auth when connecting to Elasticsearch.
-es.timeout duration
Timeout for trying to get stats from Elasticsearch. (default 5s)
-es.uri string
HTTP API address of an Elasticsearch node. (default "http://localhost:9200")
-web.listen-address string
Address to listen on for web interface and telemetry. (default ":9108")
-web.telemetry-path string
Path under which to expose metrics. (default "/metrics")
Update dashboard
Hi,
i'm trying to use this dashboard https://github.com/justwatchcom/elasticsearch_exporter/blob/master/examples/grafana/dashboard.json but it seems not working anymore.
first, if I've the node_exporter installed on all the nodes of the cluster it sums the nodes. I've 2 nodes and the nodes displayed are instead 4 since it does a sum and both nodes report 2 for that cluster.
Memory / CPU and other are not showing any data, I think (i'm not an expert) that the exported metrics have changed name.
Does anyone has a dashboard to suggest? i've seen that there are several others in grafana, but I can't get my head around to find something that show cluster + instance usage.
Error "No datadatapoints found"
I'm having an error , i open http://domain.com:9108/metrics so is full of metrics but when i open http://prometheusserver.com:9090 show metrics in graph so "No datadatapoints found"
How to fetch indices names
This is not an issue but rather a question to the team.
How do i fetch the name of indices using the elasticsearch_exported. I have properly set it up and i am able to display other metrics which are available. But i do not seem to find any metrics that can display the indices names. My use case is to display it as a table in grafana to view live data ingestion.
Binaries for v0.3.1
First of all, thanks for keeping this project! I wonder if there is any chance to release the binaries for the latest release (0.3.1)?
Thanks a lot!
Create new docker images and push them to the registry
We need to build new docker images and push them to hub.docker.com.
Maybe we should also evaluate quay.io because prometheus themselves are pushing there too.
ES_exporter doesn't export cluster statistics
Currently only node level statistics are exported. There's some valuable stats at /_cluster/health and /_cluster/stats that should be exported as well.
The key stats I noticed were missing:
- cluster status โ
- index count
- total shard count
- primary shard count
- [relocating, initializing, unassigned] shard count
/_cluster/health:
{
cluster_name: "devops-es",
status: "green",
timed_out: false,
number_of_nodes: 7,
number_of_data_nodes: 5,
active_primary_shards: 22330,
active_shards: 66990,
relocating_shards: 0,
initializing_shards: 0,
unassigned_shards: 0
}
/_cluster/stats: http://pastebin.com/1nBbEH9T
FWIW this was my reasoning behind #1 - to reduce the amount of duplicated cluster-level statistics. I'd like to implement a similar flag (--cluster-stats) to enable exporting these stats from only a subset of nodes in the cluster.
Helm chart for elasticsearch-exporter
Hi,
FYI, i've created a PR for the k8s helm chart repo to add a Helm chart for elasticsearch_exporter. (sadly reviewing takes way longer than i expected). Feel free to mention it in the installation part of the README when it's considered as stable (needs approval).
-- Sven
+elasticsearch_filesystem_data_used/free_percent
Add:
- elasticsearch_filesystem_data_used_percent
- elasticsearch_filesystem_data_free_percent
(Calculated by existing metrics)
Reason: To be able to easily add an alarm from elasticsearch_filesystem_data_used_percent metric.
Please document how often does exporter pull ES cluster state
It will be good to have a documentation about how often do you poll ES cluster state. Is it on every scrape or just periodically? If the latter, can I control the interval?
Thanks.
How to user your code?
Hi,can you give me an example to use your code to monitor es server? tanks
failed run exporter inside docker
Hello,
I try to ran image inside docker build :
- Dockerfile
FROM golang
RUN go get -u github.com/justwatchcom/elasticsearch_exporter
COPY start.sh /
RUN chmod +x /start.sh
CMD /start.sh
- start.sh
#!/bin/bash
elasticsearch_exporter -es.uri http://my.ip.ad.dr:9200
- docker-compose.yml
prom-elastic:
build: ./prom-elastic
container_name: prom-elastic
hostname: prom-elastic
ports:
- 9108:9108
- docker version
# docker --version
Docker version 17.03.1-ce, build c6d412e
Error :
2017/06/12 20:45:58 Starting Server: :9108
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x6f8062]
goroutine 49 [running]:
github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/client_golang/prometheus.(*CounterVec).WithLabelValues(0x0, 0xc42009c938, 0x2, 0x2, 0xa77300, 0xc42027b200)
/go/src/github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/client_golang/prometheus/counter.go:139 +0x22
main.(*Exporter).Collect(0xc420194080, 0xc420196480)
/go/src/github.com/justwatchcom/elasticsearch_exporter/exporter.go:459 +0x39af
github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/client_golang/prometheus.(*registry).writePB.func2(0xc4201b4750, 0xc420196480, 0xa252e0, 0xc420194080)
/go/src/github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/client_golang/prometheus/registry.go:420 +0x61
created by github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/client_golang/prometheus.(*registry).writePB
/go/src/github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/client_golang/prometheus/registry.go:421 +0x274
if basic auth is used, please remove user/password from err
Hi,
it would be nice to obfuscate the username and password, when basic auth is used in the es.url. Right now on level warn the url with password us displayed in the logs. This is relevant if you the exporter on kubernetes and share the kibana instance.
Thanks a lot and regards,
Olaf
tutorial on how to start from scratch
Hello,
I've several nodes running elastic, and I would like to install the exporter on each node.
Now, I've installed GO, downloading the package and running tar -C /usr/local -xzf go$VERSION.$OS-$ARCH.tar.gz adding export PATH=$PATH:/usr/local/go/bin as said here https://golang.org/doc/install
Then, following the readme i run go get -u github.com/justwatchcom/elasticsearch_exporter which ends without saying anything (is that fine?).
And, when I try to run elasticsearch_exporter --help i get a command not found .
what's the problem? i'm quite new to go and how it works..
When you will release next version?
As the title.
I found the code updated recently, contain a lot of useful metrics, do you have any plan for next releasing?
Prefix for index-level metrics
I am going to implement index-level metrics using the indices API. As I can see, the "indices" metric prefix is already used for metrics obtained via /_nodes/stats - for example, indices_filter_cache_memory_size_bytes.
I guess renaming the metrics is not an option, since multiple people are already using the exporter. I was thinking about using "index", i.e.:
index_filter_cache_memory_size_byteswill haveclusterandindexlabelsindices_filter_cache_memory_size_byteswill remain as it currently is, withclusterandnodelabels
What do you think?
Missing Metrics
I've set this up with the following kubernetes container spec:
- name: elasticsearch-exporter
image: justwatch/elasticsearch_exporter:1.0.1
args:
- --es.uri=http://localhost:9200
- --es.all=true
ports:
- containerPort: 9108
name: es-expShould I be doing anything else to get all the metrics?
The metrics I'm seeing in prometheus don't match with the metrics listed in the readme.
elasticsearch_cluster_health_json_parse_failures
elasticsearch_cluster_health_total_scrapes
elasticsearch_cluster_health_up
elasticsearch_node_health_json_parse_failures
elasticsearch_node_health_total_scrapes
elasticsearch_node_health_up
I'm using elasticsearch version 2.4.1
Label node returns IP instead of hostname
On ES 1.x it used to return:
elasticsearch_thread_pool_threads_count{cluster="monitoring",node="elasticsearch-data-n1-v211-p0-b7707",type="get"}
Now on ES 2.1.1:
elasticsearch_thread_pool_threads_count{cluster="monitoring",node="10.244.6.19",type="get"}
ElasticSearch /_nodes output:
{
"cluster_name" : "monitoring",
"nodes" : {
"3V6hkcaOQZyTtGpun7-viw" : {
"name" : "elasticsearch-data-n2-v211-p0-jhxix",
"transport_address" : "10.244.6.20:9300",
"host" : "10.244.6.20",
"ip" : "10.244.6.20",
"version" : "2.1.1",
"build" : "40e2c53",
"http_address" : "10.244.6.20:9200",
"attributes" : {
"master" : "false"
},
"settings" : {
"cluster" : {
"name" : "monitoring"
},
"index" : {
"translog" : {
"flush_threshold_ops" : "50000"
}
},
"bootstrap" : {
"mlockall" : "false"
},
"script" : {
"inline" : "sandbox",
"indexed" : "sandbox",
"file" : "off"
},
"network" : {
"host" : "0.0.0.0"
},
"cloud" : {
"kubernetes" : {
"namespace" : "default",
"service" : "elasticsearch-cluster"
}
},
"node" : {
"data" : "true",
"name" : "elasticsearch-data-n2-v211-p0-jhxix",
"master" : "false"
},
"path" : {
"data" : "/elasticsearch/data",
"logs" : "/elasticsearch/log",
"plugins" : "/elasticsearch/plugins",
"work" : "/elasticsearch/work",
"home" : "/elasticsearch"
},
"indices" : {
"cache" : {
"filter" : {
"size" : "30%"
}
},
"memory" : {
"index_buffer_size" : "20%"
},
"recovery" : {
"concurrent_streams" : "12",
"translog_size" : "2mb",
"translog_ops" : "4000",
"max_bytes_per_sec" : "60mb",
"file_chunk_size" : "2mb"
},
"store" : {
"throttle" : {
"max_bytes_per_sec" : "40mb"
}
},
"breaker" : {
"fielddata" : {
"limit" : "3584mb"
}
},
"fielddata" : {
"cache" : {
"size" : "3072mb"
}
}
},
"security" : {
"manager" : {
"enabled" : "false"
}
},
"marvel" : {
"agent" : {
"enabled" : "false"
}
},
"discovery" : {
"type" : "kubernetes",
"zen" : {
"ping" : {
"multicast" : {
"enabled" : "false"
}
}
}
},
"merge" : {
"policy" : {
"merge_factor" : "30"
}
},
"name" : "elasticsearch-data-n2-v211-p0-jhxix",
"action" : {
"disable_delete_all_indices" : "true"
},
"client" : {
"type" : "node"
},
"http" : {
"enabled" : "true",
"max_content_length" : "100mb"
},
"config" : {
"ignore_system_properties" : "true"
}
},
"os" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"available_processors" : 4,
"allocated_processors" : 4
},
"process" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"id" : 12,
"mlockall" : false
},
"jvm" : {
"pid" : 12,
"version" : "1.8.0_66",
"vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
"vm_version" : "25.66-b17",
"vm_vendor" : "Oracle Corporation",
"start_time" : "2016-01-19T20:38:00.327Z",
"start_time_in_millis" : 1453235880327,
"mem" : {
"heap_init" : "4gb",
"heap_init_in_bytes" : 4294967296,
"heap_max" : "3.9gb",
"heap_max_in_bytes" : 4260102144,
"non_heap_init" : "2.4mb",
"non_heap_init_in_bytes" : 2555904,
"non_heap_max" : "0b",
"non_heap_max_in_bytes" : 0,
"direct_max" : "3.9gb",
"direct_max_in_bytes" : 4260102144
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen" ]
},
"thread_pool" : {
"force_merge" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"fetch_shard_started" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"listener" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 200
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 7,
"max" : 7,
"queue_size" : 1000
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"fetch_shard_store" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"bulk" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 50
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"transport" : {
"bound_address" : [ "[::]:9300" ],
"publish_address" : "10.244.6.20:9300",
"profiles" : { }
},
"http" : {
"bound_address" : [ "[::]:9200" ],
"publish_address" : "10.244.6.20:9200",
"max_content_length" : "100mb",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ {
"name" : "analysis-phonetic",
"version" : "2.1.1",
"description" : "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "cloud-kubernetes",
"version" : "2.1.1",
"description" : "Elasticsearch Kubernetes cloud plugin",
"jvm" : true,
"classname" : "io.fabric8.elasticsearch.plugin.discovery.kubernetes.KubernetesDiscoveryPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "hq",
"version" : "1.0.0",
"description" : "Elastic-HQ",
"url" : "/_plugin/hq/",
"jvm" : false,
"site" : true
}, {
"name" : "lang-javascript",
"version" : "2.1.1",
"description" : "The JavaScript language plugin allows to have javascript as the language of scripts to execute.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.javascript.JavaScriptPlugin",
"isolated" : true,
"site" : false
} ]
},
"zzUlKYapSYyieT530W787A" : {
"name" : "elasticsearch-master-n0-v211-p0-sb10d",
"transport_address" : "10.244.3.33:9300",
"host" : "10.244.3.33",
"ip" : "10.244.3.33",
"version" : "2.1.1",
"build" : "40e2c53",
"http_address" : "10.244.3.33:9200",
"attributes" : {
"data" : "false",
"master" : "true"
},
"settings" : {
"cluster" : {
"name" : "monitoring"
},
"index" : {
"translog" : {
"flush_threshold_ops" : "50000"
}
},
"bootstrap" : {
"mlockall" : "false"
},
"script" : {
"inline" : "sandbox",
"indexed" : "sandbox",
"file" : "off"
},
"network" : {
"host" : "0.0.0.0"
},
"cloud" : {
"kubernetes" : {
"namespace" : "default",
"service" : "elasticsearch-cluster"
}
},
"node" : {
"data" : "false",
"name" : "elasticsearch-master-n0-v211-p0-sb10d",
"master" : "true"
},
"path" : {
"data" : "/elasticsearch/data",
"logs" : "/elasticsearch/log",
"plugins" : "/elasticsearch/plugins",
"work" : "/elasticsearch/work",
"home" : "/elasticsearch"
},
"indices" : {
"cache" : {
"filter" : {
"size" : "30%"
}
},
"memory" : {
"index_buffer_size" : "20%"
},
"recovery" : {
"concurrent_streams" : "12",
"translog_size" : "2mb",
"translog_ops" : "4000",
"max_bytes_per_sec" : "60mb",
"file_chunk_size" : "2mb"
},
"store" : {
"throttle" : {
"max_bytes_per_sec" : "40mb"
}
},
"breaker" : {
"fielddata" : {
"limit" : "3584mb"
}
},
"fielddata" : {
"cache" : {
"size" : "3072mb"
}
}
},
"security" : {
"manager" : {
"enabled" : "false"
}
},
"marvel" : {
"agent" : {
"enabled" : "false"
}
},
"discovery" : {
"type" : "kubernetes",
"zen" : {
"ping" : {
"multicast" : {
"enabled" : "false"
}
}
}
},
"merge" : {
"policy" : {
"merge_factor" : "30"
}
},
"name" : "elasticsearch-master-n0-v211-p0-sb10d",
"action" : {
"disable_delete_all_indices" : "true"
},
"client" : {
"type" : "node"
},
"http" : {
"enabled" : "true",
"max_content_length" : "100mb"
},
"config" : {
"ignore_system_properties" : "true"
}
},
"os" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"available_processors" : 4,
"allocated_processors" : 4
},
"process" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"id" : 14,
"mlockall" : false
},
"jvm" : {
"pid" : 14,
"version" : "1.8.0_66",
"vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
"vm_version" : "25.66-b17",
"vm_vendor" : "Oracle Corporation",
"start_time" : "2016-01-19T20:33:33.053Z",
"start_time_in_millis" : 1453235613053,
"mem" : {
"heap_init" : "4gb",
"heap_init_in_bytes" : 4294967296,
"heap_max" : "3.9gb",
"heap_max_in_bytes" : 4260102144,
"non_heap_init" : "2.4mb",
"non_heap_init_in_bytes" : 2555904,
"non_heap_max" : "0b",
"non_heap_max_in_bytes" : 0,
"direct_max" : "3.9gb",
"direct_max_in_bytes" : 4260102144
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen" ]
},
"thread_pool" : {
"force_merge" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"fetch_shard_started" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"listener" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 200
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 7,
"max" : 7,
"queue_size" : 1000
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"fetch_shard_store" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"bulk" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 50
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"transport" : {
"bound_address" : [ "[::]:9300" ],
"publish_address" : "10.244.3.33:9300",
"profiles" : { }
},
"http" : {
"bound_address" : [ "[::]:9200" ],
"publish_address" : "10.244.3.33:9200",
"max_content_length" : "100mb",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ {
"name" : "analysis-phonetic",
"version" : "2.1.1",
"description" : "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "cloud-kubernetes",
"version" : "2.1.1",
"description" : "Elasticsearch Kubernetes cloud plugin",
"jvm" : true,
"classname" : "io.fabric8.elasticsearch.plugin.discovery.kubernetes.KubernetesDiscoveryPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "hq",
"version" : "1.0.0",
"description" : "Elastic-HQ",
"url" : "/_plugin/hq/",
"jvm" : false,
"site" : true
}, {
"name" : "lang-javascript",
"version" : "2.1.1",
"description" : "The JavaScript language plugin allows to have javascript as the language of scripts to execute.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.javascript.JavaScriptPlugin",
"isolated" : true,
"site" : false
} ]
},
"lGMhCrcyR8qMjSRhKd4vUw" : {
"name" : "elasticsearch-data-n3-v211-p0-2qcp6",
"transport_address" : "10.244.0.15:9300",
"host" : "10.244.0.15",
"ip" : "10.244.0.15",
"version" : "2.1.1",
"build" : "40e2c53",
"http_address" : "10.244.0.15:9200",
"attributes" : {
"master" : "false"
},
"settings" : {
"cluster" : {
"name" : "monitoring"
},
"index" : {
"translog" : {
"flush_threshold_ops" : "50000"
}
},
"bootstrap" : {
"mlockall" : "false"
},
"script" : {
"inline" : "sandbox",
"indexed" : "sandbox",
"file" : "off"
},
"network" : {
"host" : "0.0.0.0"
},
"cloud" : {
"kubernetes" : {
"namespace" : "default",
"service" : "elasticsearch-cluster"
}
},
"node" : {
"data" : "true",
"name" : "elasticsearch-data-n3-v211-p0-2qcp6",
"master" : "false"
},
"path" : {
"data" : "/elasticsearch/data",
"logs" : "/elasticsearch/log",
"plugins" : "/elasticsearch/plugins",
"work" : "/elasticsearch/work",
"home" : "/elasticsearch"
},
"indices" : {
"cache" : {
"filter" : {
"size" : "30%"
}
},
"memory" : {
"index_buffer_size" : "20%"
},
"recovery" : {
"concurrent_streams" : "12",
"translog_size" : "2mb",
"translog_ops" : "4000",
"max_bytes_per_sec" : "60mb",
"file_chunk_size" : "2mb"
},
"store" : {
"throttle" : {
"max_bytes_per_sec" : "40mb"
}
},
"breaker" : {
"fielddata" : {
"limit" : "3584mb"
}
},
"fielddata" : {
"cache" : {
"size" : "3072mb"
}
}
},
"security" : {
"manager" : {
"enabled" : "false"
}
},
"marvel" : {
"agent" : {
"enabled" : "false"
}
},
"discovery" : {
"type" : "kubernetes",
"zen" : {
"ping" : {
"multicast" : {
"enabled" : "false"
}
}
}
},
"merge" : {
"policy" : {
"merge_factor" : "30"
}
},
"name" : "elasticsearch-data-n3-v211-p0-2qcp6",
"action" : {
"disable_delete_all_indices" : "true"
},
"client" : {
"type" : "node"
},
"http" : {
"enabled" : "true",
"max_content_length" : "100mb"
},
"config" : {
"ignore_system_properties" : "true"
}
},
"os" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"available_processors" : 4,
"allocated_processors" : 4
},
"process" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"id" : 17,
"mlockall" : false
},
"jvm" : {
"pid" : 17,
"version" : "1.8.0_66",
"vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
"vm_version" : "25.66-b17",
"vm_vendor" : "Oracle Corporation",
"start_time" : "2016-01-19T20:37:47.028Z",
"start_time_in_millis" : 1453235867028,
"mem" : {
"heap_init" : "4gb",
"heap_init_in_bytes" : 4294967296,
"heap_max" : "3.9gb",
"heap_max_in_bytes" : 4260102144,
"non_heap_init" : "2.4mb",
"non_heap_init_in_bytes" : 2555904,
"non_heap_max" : "0b",
"non_heap_max_in_bytes" : 0,
"direct_max" : "3.9gb",
"direct_max_in_bytes" : 4260102144
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen" ]
},
"thread_pool" : {
"force_merge" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"fetch_shard_started" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"listener" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 200
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 7,
"max" : 7,
"queue_size" : 1000
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"fetch_shard_store" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"bulk" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 50
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"transport" : {
"bound_address" : [ "[::]:9300" ],
"publish_address" : "10.244.0.15:9300",
"profiles" : { }
},
"http" : {
"bound_address" : [ "[::]:9200" ],
"publish_address" : "10.244.0.15:9200",
"max_content_length" : "100mb",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ {
"name" : "analysis-phonetic",
"version" : "2.1.1",
"description" : "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "cloud-kubernetes",
"version" : "2.1.1",
"description" : "Elasticsearch Kubernetes cloud plugin",
"jvm" : true,
"classname" : "io.fabric8.elasticsearch.plugin.discovery.kubernetes.KubernetesDiscoveryPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "hq",
"version" : "1.0.0",
"description" : "Elastic-HQ",
"url" : "/_plugin/hq/",
"jvm" : false,
"site" : true
}, {
"name" : "lang-javascript",
"version" : "2.1.1",
"description" : "The JavaScript language plugin allows to have javascript as the language of scripts to execute.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.javascript.JavaScriptPlugin",
"isolated" : true,
"site" : false
} ]
},
"jIypga7CTWm8u4nuMFxc_w" : {
"name" : "elasticsearch-data-n0-v211-p0-xe2mi",
"transport_address" : "10.244.5.18:9300",
"host" : "10.244.5.18",
"ip" : "10.244.5.18",
"version" : "2.1.1",
"build" : "40e2c53",
"http_address" : "10.244.5.18:9200",
"attributes" : {
"master" : "false"
},
"settings" : {
"cluster" : {
"name" : "monitoring"
},
"index" : {
"translog" : {
"flush_threshold_ops" : "50000"
}
},
"bootstrap" : {
"mlockall" : "false"
},
"script" : {
"inline" : "sandbox",
"indexed" : "sandbox",
"file" : "off"
},
"network" : {
"host" : "0.0.0.0"
},
"cloud" : {
"kubernetes" : {
"namespace" : "default",
"service" : "elasticsearch-cluster"
}
},
"node" : {
"data" : "true",
"name" : "elasticsearch-data-n0-v211-p0-xe2mi",
"master" : "false"
},
"path" : {
"data" : "/elasticsearch/data",
"logs" : "/elasticsearch/log",
"plugins" : "/elasticsearch/plugins",
"work" : "/elasticsearch/work",
"home" : "/elasticsearch"
},
"indices" : {
"cache" : {
"filter" : {
"size" : "30%"
}
},
"memory" : {
"index_buffer_size" : "20%"
},
"recovery" : {
"concurrent_streams" : "12",
"translog_size" : "2mb",
"translog_ops" : "4000",
"max_bytes_per_sec" : "60mb",
"file_chunk_size" : "2mb"
},
"store" : {
"throttle" : {
"max_bytes_per_sec" : "40mb"
}
},
"breaker" : {
"fielddata" : {
"limit" : "3584mb"
}
},
"fielddata" : {
"cache" : {
"size" : "3072mb"
}
}
},
"security" : {
"manager" : {
"enabled" : "false"
}
},
"marvel" : {
"agent" : {
"enabled" : "false"
}
},
"discovery" : {
"type" : "kubernetes",
"zen" : {
"ping" : {
"multicast" : {
"enabled" : "false"
}
}
}
},
"merge" : {
"policy" : {
"merge_factor" : "30"
}
},
"name" : "elasticsearch-data-n0-v211-p0-xe2mi",
"action" : {
"disable_delete_all_indices" : "true"
},
"client" : {
"type" : "node"
},
"http" : {
"enabled" : "true",
"max_content_length" : "100mb"
},
"config" : {
"ignore_system_properties" : "true"
}
},
"os" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"available_processors" : 4,
"allocated_processors" : 4
},
"process" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"id" : 14,
"mlockall" : false
},
"jvm" : {
"pid" : 14,
"version" : "1.8.0_66",
"vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
"vm_version" : "25.66-b17",
"vm_vendor" : "Oracle Corporation",
"start_time" : "2016-01-19T20:37:45.184Z",
"start_time_in_millis" : 1453235865184,
"mem" : {
"heap_init" : "4gb",
"heap_init_in_bytes" : 4294967296,
"heap_max" : "3.9gb",
"heap_max_in_bytes" : 4260102144,
"non_heap_init" : "2.4mb",
"non_heap_init_in_bytes" : 2555904,
"non_heap_max" : "0b",
"non_heap_max_in_bytes" : 0,
"direct_max" : "3.9gb",
"direct_max_in_bytes" : 4260102144
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen" ]
},
"thread_pool" : {
"force_merge" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"fetch_shard_started" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"listener" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 200
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 7,
"max" : 7,
"queue_size" : 1000
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"fetch_shard_store" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"bulk" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 50
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"transport" : {
"bound_address" : [ "[::]:9300" ],
"publish_address" : "10.244.5.18:9300",
"profiles" : { }
},
"http" : {
"bound_address" : [ "[::]:9200" ],
"publish_address" : "10.244.5.18:9200",
"max_content_length" : "100mb",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ {
"name" : "analysis-phonetic",
"version" : "2.1.1",
"description" : "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "cloud-kubernetes",
"version" : "2.1.1",
"description" : "Elasticsearch Kubernetes cloud plugin",
"jvm" : true,
"classname" : "io.fabric8.elasticsearch.plugin.discovery.kubernetes.KubernetesDiscoveryPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "hq",
"version" : "1.0.0",
"description" : "Elastic-HQ",
"url" : "/_plugin/hq/",
"jvm" : false,
"site" : true
}, {
"name" : "lang-javascript",
"version" : "2.1.1",
"description" : "The JavaScript language plugin allows to have javascript as the language of scripts to execute.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.javascript.JavaScriptPlugin",
"isolated" : true,
"site" : false
} ]
},
"MhkpsIvnQ-G9r6g_6V3pHA" : {
"name" : "elasticsearch-data-n1-v211-p0-b7707",
"transport_address" : "10.244.4.15:9300",
"host" : "10.244.4.15",
"ip" : "10.244.4.15",
"version" : "2.1.1",
"build" : "40e2c53",
"http_address" : "10.244.4.15:9200",
"attributes" : {
"master" : "false"
},
"settings" : {
"cluster" : {
"name" : "monitoring"
},
"index" : {
"translog" : {
"flush_threshold_ops" : "50000"
}
},
"bootstrap" : {
"mlockall" : "false"
},
"script" : {
"inline" : "sandbox",
"indexed" : "sandbox",
"file" : "off"
},
"network" : {
"host" : "0.0.0.0"
},
"cloud" : {
"kubernetes" : {
"namespace" : "default",
"service" : "elasticsearch-cluster"
}
},
"node" : {
"data" : "true",
"name" : "elasticsearch-data-n1-v211-p0-b7707",
"master" : "false"
},
"path" : {
"data" : "/elasticsearch/data",
"logs" : "/elasticsearch/log",
"plugins" : "/elasticsearch/plugins",
"work" : "/elasticsearch/work",
"home" : "/elasticsearch"
},
"indices" : {
"cache" : {
"filter" : {
"size" : "30%"
}
},
"memory" : {
"index_buffer_size" : "20%"
},
"recovery" : {
"concurrent_streams" : "12",
"translog_size" : "2mb",
"translog_ops" : "4000",
"max_bytes_per_sec" : "60mb",
"file_chunk_size" : "2mb"
},
"store" : {
"throttle" : {
"max_bytes_per_sec" : "40mb"
}
},
"breaker" : {
"fielddata" : {
"limit" : "3584mb"
}
},
"fielddata" : {
"cache" : {
"size" : "3072mb"
}
}
},
"security" : {
"manager" : {
"enabled" : "false"
}
},
"marvel" : {
"agent" : {
"enabled" : "false"
}
},
"discovery" : {
"type" : "kubernetes",
"zen" : {
"ping" : {
"multicast" : {
"enabled" : "false"
}
}
}
},
"merge" : {
"policy" : {
"merge_factor" : "30"
}
},
"name" : "elasticsearch-data-n1-v211-p0-b7707",
"action" : {
"disable_delete_all_indices" : "true"
},
"client" : {
"type" : "node"
},
"http" : {
"enabled" : "true",
"max_content_length" : "100mb"
},
"config" : {
"ignore_system_properties" : "true"
}
},
"os" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"available_processors" : 4,
"allocated_processors" : 4
},
"process" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"id" : 11,
"mlockall" : false
},
"jvm" : {
"pid" : 11,
"version" : "1.8.0_66",
"vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
"vm_version" : "25.66-b17",
"vm_vendor" : "Oracle Corporation",
"start_time" : "2016-01-19T20:37:45.760Z",
"start_time_in_millis" : 1453235865760,
"mem" : {
"heap_init" : "4gb",
"heap_init_in_bytes" : 4294967296,
"heap_max" : "3.9gb",
"heap_max_in_bytes" : 4260102144,
"non_heap_init" : "2.4mb",
"non_heap_init_in_bytes" : 2555904,
"non_heap_max" : "0b",
"non_heap_max_in_bytes" : 0,
"direct_max" : "3.9gb",
"direct_max_in_bytes" : 4260102144
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen" ]
},
"thread_pool" : {
"force_merge" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"fetch_shard_started" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"listener" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 200
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 7,
"max" : 7,
"queue_size" : 1000
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"fetch_shard_store" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"bulk" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 50
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"transport" : {
"bound_address" : [ "[::]:9300" ],
"publish_address" : "10.244.4.15:9300",
"profiles" : { }
},
"http" : {
"bound_address" : [ "[::]:9200" ],
"publish_address" : "10.244.4.15:9200",
"max_content_length" : "100mb",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ {
"name" : "analysis-phonetic",
"version" : "2.1.1",
"description" : "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "cloud-kubernetes",
"version" : "2.1.1",
"description" : "Elasticsearch Kubernetes cloud plugin",
"jvm" : true,
"classname" : "io.fabric8.elasticsearch.plugin.discovery.kubernetes.KubernetesDiscoveryPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "hq",
"version" : "1.0.0",
"description" : "Elastic-HQ",
"url" : "/_plugin/hq/",
"jvm" : false,
"site" : true
}, {
"name" : "lang-javascript",
"version" : "2.1.1",
"description" : "The JavaScript language plugin allows to have javascript as the language of scripts to execute.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.javascript.JavaScriptPlugin",
"isolated" : true,
"site" : false
} ]
},
"1MoYzsloS0GYO88ReaGd9Q" : {
"name" : "elasticsearch-lb-v211-p0-bqbni",
"transport_address" : "10.244.2.12:9300",
"host" : "10.244.2.12",
"ip" : "10.244.2.12",
"version" : "2.1.1",
"build" : "40e2c53",
"http_address" : "10.244.2.12:9200",
"attributes" : {
"data" : "false",
"master" : "false"
},
"settings" : {
"cluster" : {
"name" : "monitoring"
},
"index" : {
"translog" : {
"flush_threshold_ops" : "50000"
}
},
"bootstrap" : {
"mlockall" : "false"
},
"script" : {
"inline" : "sandbox",
"indexed" : "sandbox",
"file" : "off"
},
"network" : {
"host" : "0.0.0.0"
},
"cloud" : {
"kubernetes" : {
"namespace" : "default",
"service" : "elasticsearch-cluster"
}
},
"node" : {
"data" : "false",
"name" : "elasticsearch-lb-v211-p0-bqbni",
"master" : "false"
},
"path" : {
"data" : "/elasticsearch/data",
"logs" : "/elasticsearch/log",
"plugins" : "/elasticsearch/plugins",
"work" : "/elasticsearch/work",
"home" : "/elasticsearch"
},
"indices" : {
"cache" : {
"filter" : {
"size" : "30%"
}
},
"memory" : {
"index_buffer_size" : "20%"
},
"recovery" : {
"concurrent_streams" : "12",
"translog_size" : "2mb",
"translog_ops" : "4000",
"max_bytes_per_sec" : "60mb",
"file_chunk_size" : "2mb"
},
"store" : {
"throttle" : {
"max_bytes_per_sec" : "40mb"
}
},
"breaker" : {
"fielddata" : {
"limit" : "3584mb"
}
},
"fielddata" : {
"cache" : {
"size" : "3072mb"
}
}
},
"security" : {
"manager" : {
"enabled" : "false"
}
},
"marvel" : {
"agent" : {
"enabled" : "false"
}
},
"discovery" : {
"type" : "kubernetes",
"zen" : {
"ping" : {
"multicast" : {
"enabled" : "false"
}
}
}
},
"merge" : {
"policy" : {
"merge_factor" : "30"
}
},
"name" : "elasticsearch-lb-v211-p0-bqbni",
"action" : {
"disable_delete_all_indices" : "true"
},
"client" : {
"type" : "node"
},
"http" : {
"enabled" : "true",
"max_content_length" : "100mb"
},
"config" : {
"ignore_system_properties" : "true"
}
},
"os" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"name" : "Linux",
"arch" : "amd64",
"version" : "3.16.0-0.bpo.4-amd64",
"available_processors" : 4,
"allocated_processors" : 4
},
"process" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"id" : 12,
"mlockall" : false
},
"jvm" : {
"pid" : 12,
"version" : "1.8.0_66",
"vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
"vm_version" : "25.66-b17",
"vm_vendor" : "Oracle Corporation",
"start_time" : "2016-01-19T20:38:42.245Z",
"start_time_in_millis" : 1453235922245,
"mem" : {
"heap_init" : "4gb",
"heap_init_in_bytes" : 4294967296,
"heap_max" : "3.9gb",
"heap_max_in_bytes" : 4260102144,
"non_heap_init" : "2.4mb",
"non_heap_init_in_bytes" : 2555904,
"non_heap_max" : "0b",
"non_heap_max_in_bytes" : 0,
"direct_max" : "3.9gb",
"direct_max_in_bytes" : 4260102144
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen" ]
},
"thread_pool" : {
"force_merge" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"fetch_shard_started" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"listener" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 200
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 7,
"max" : 7,
"queue_size" : 1000
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"fetch_shard_store" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"bulk" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 50
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"transport" : {
"bound_address" : [ "[::]:9300" ],
"publish_address" : "10.244.2.12:9300",
"profiles" : { }
},
"http" : {
"bound_address" : [ "[::]:9200" ],
"publish_address" : "10.244.2.12:9200",
"max_content_length" : "100mb",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ {
"name" : "analysis-phonetic",
"version" : "2.1.1",
"description" : "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "cloud-kubernetes",
"version" : "2.1.1",
"description" : "Elasticsearch Kubernetes cloud plugin",
"jvm" : true,
"classname" : "io.fabric8.elasticsearch.plugin.discovery.kubernetes.KubernetesDiscoveryPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "hq",
"version" : "1.0.0",
"description" : "Elastic-HQ",
"url" : "/_plugin/hq/",
"jvm" : false,
"site" : true
}, {
"name" : "lang-javascript",
"version" : "2.1.1",
"description" : "The JavaScript language plugin allows to have javascript as the language of scripts to execute.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.javascript.JavaScriptPlugin",
"isolated" : true,
"site" : false
} ]
},
"m-ZtgY8OQYyfaCyJErYEgA" : {
"name" : "elasticsearch-master-n1-v211-p0-0qzhd",
"transport_address" : "10.244.6.19:9300",
"host" : "10.244.6.19",
"ip" : "10.244.6.19",
"version" : "2.1.1",
"build" : "40e2c53",
"http_address" : "10.244.6.19:9200",
"attributes" : {
"data" : "false",
"master" : "true"
},
"settings" : {
"cluster" : {
"name" : "monitoring"
},
"index" : {
"translog" : {
"flush_threshold_ops" : "50000"
}
},
"bootstrap" : {
"mlockall" : "false"
},
"script" : {
"inline" : "sandbox",
"indexed" : "sandbox",
"file" : "off"
},
"network" : {
"host" : "0.0.0.0"
},
"cloud" : {
"kubernetes" : {
"namespace" : "default",
"service" : "elasticsearch-cluster"
}
},
"node" : {
"data" : "false",
"name" : "elasticsearch-master-n1-v211-p0-0qzhd",
"master" : "true"
},
"path" : {
"data" : "/elasticsearch/data",
"logs" : "/elasticsearch/log",
"plugins" : "/elasticsearch/plugins",
"work" : "/elasticsearch/work",
"home" : "/elasticsearch"
},
"indices" : {
"cache" : {
"filter" : {
"size" : "30%"
}
},
"memory" : {
"index_buffer_size" : "20%"
},
"recovery" : {
"concurrent_streams" : "12",
"translog_size" : "2mb",
"translog_ops" : "4000",
"max_bytes_per_sec" : "60mb",
"file_chunk_size" : "2mb"
},
"store" : {
"throttle" : {
"max_bytes_per_sec" : "40mb"
}
},
"breaker" : {
"fielddata" : {
"limit" : "3584mb"
}
},
"fielddata" : {
"cache" : {
"size" : "3072mb"
}
}
},
"security" : {
"manager" : {
"enabled" : "false"
}
},
"marvel" : {
"agent" : {
"enabled" : "false"
}
},
"discovery" : {
"type" : "kubernetes",
"zen" : {
"ping" : {
"multicast" : {
"enabled" : "false"
}
}
}
},
"merge" : {
"policy" : {
"merge_factor" : "30"
}
},
"name" : "elasticsearch-master-n1-v211-p0-0qzhd",
"action" : {
"disable_delete_all_indices" : "true"
},
"client" : {
"type" : "node"
},
"http" : {
"enabled" : "true",
"max_content_length" : "100mb"
},
"config" : {
"ignore_system_properties" : "true"
}
},
"os" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"available_processors" : 4,
"allocated_processors" : 4
},
"process" : {
"refresh_interval" : "1s",
"refresh_interval_in_millis" : 1000,
"id" : 15,
"mlockall" : false
},
"jvm" : {
"pid" : 15,
"version" : "1.8.0_66",
"vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
"vm_version" : "25.66-b17",
"vm_vendor" : "Oracle Corporation",
"start_time" : "2016-01-19T20:30:29.170Z",
"start_time_in_millis" : 1453235429170,
"mem" : {
"heap_init" : "4gb",
"heap_init_in_bytes" : 4294967296,
"heap_max" : "3.9gb",
"heap_max_in_bytes" : 4260102144,
"non_heap_init" : "2.4mb",
"non_heap_init_in_bytes" : 2555904,
"non_heap_max" : "0b",
"non_heap_max_in_bytes" : 0,
"direct_max" : "3.9gb",
"direct_max_in_bytes" : 4260102144
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen" ]
},
"thread_pool" : {
"force_merge" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"fetch_shard_started" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"listener" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 200
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 7,
"max" : 7,
"queue_size" : 1000
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
},
"fetch_shard_store" : {
"type" : "scaling",
"min" : 1,
"max" : 8,
"keep_alive" : "5m",
"queue_size" : -1
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 1000
},
"bulk" : {
"type" : "fixed",
"min" : 4,
"max" : 4,
"queue_size" : 50
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 2,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"transport" : {
"bound_address" : [ "[::]:9300" ],
"publish_address" : "10.244.6.19:9300",
"profiles" : { }
},
"http" : {
"bound_address" : [ "[::]:9200" ],
"publish_address" : "10.244.6.19:9200",
"max_content_length" : "100mb",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ {
"name" : "analysis-phonetic",
"version" : "2.1.1",
"description" : "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "cloud-kubernetes",
"version" : "2.1.1",
"description" : "Elasticsearch Kubernetes cloud plugin",
"jvm" : true,
"classname" : "io.fabric8.elasticsearch.plugin.discovery.kubernetes.KubernetesDiscoveryPlugin",
"isolated" : true,
"site" : false
}, {
"name" : "hq",
"version" : "1.0.0",
"description" : "Elastic-HQ",
"url" : "/_plugin/hq/",
"jvm" : false,
"site" : true
}, {
"name" : "lang-javascript",
"version" : "2.1.1",
"description" : "The JavaScript language plugin allows to have javascript as the language of scripts to execute.",
"jvm" : true,
"classname" : "org.elasticsearch.plugin.javascript.JavaScriptPlugin",
"isolated" : true,
"site" : false
} ]
}
}
}
elasticsearch_up metric
I'm using elasticsearch_up metric to display elasticsearch status in a grafana dashboard. However, for reasons that I do not understand, this metric value is constantly switching from 0 to 1 and back to 0 IF queried using the prometheus interface. Accessing the exporter metrics endpoint directly, elasticsearch_up metric allways returns the value 1!
I'm using haproxy exporter's haproxy_up metric and it works without any problem. I have no ideia what's causing this behavoir.
Any idea what might be happening here?
Time metrics use Milliseconds instead of base unit - seconds
Great work on this exporter! we are using it with great results ๐
One small comment, the following metrics use Milliseconds instead of a base unit (seconds)
elasticsearch_indices_flush_time_ms_total
elasticsearch_indices_indexing_index_time_ms_tot
elasticsearch_indices_merges_total_time_ms_total
elasticsearch_indices_refresh_total_time_ms_total
elasticsearch_indices_store_throttle_time_ms_total
elasticsearch_indices_search_query_time_ms
Is there any specific reason to use ms instead of sec?
I know I'm probably nit-picking here, but was reviewing the best practices from Prometheus and they seem to suggest to use base units, and leave handling the presentation in the Dashboarding tool (i.e. Grafana).
...should use base units (e.g. seconds, bytes, meters - not milliseconds, megabytes, kilometers).
https://prometheus.io/docs/practices/naming/
Any thoughts?
Add a migration section to the readme for updating from ewr/elasticsearch_exporter
Title says it all.
Probably just say to update the name of the docker image on the deployments.
Monitoring if Elasticsearch is Down
Hi there,
I'm trying to monitor my elasticsearch instance so that if it's down an alert is triggered.
I created this basic rule -
# Alert if elasticsearch is unreachable for >1 minute.
ALERT ElasticSearchDown
IF elasticsearch_up == 0
FOR 1m
WITH {
severity="page"
}
SUMMARY "Instance {{$labels.instance}} elasticsearch is down."
DESCRIPTION "{{$labels.instance}} of job {{$labels.job}} has been down for more than 1 minutes."
But the issue I'm facing is that if I issue a shutdown using the following command -
curl -XPOST 'http://localhost:9200/_cluster/nodes/_local/_shutdown'
Instead of the metrics reflecting that Elasticsearch is down - all the metrics disappear!
Is there a better way to monitor that Elasticsearch is running and shouldn't the metric 'elasticsearch_up' == 0 if the elasticsearch_exporter cannot reach Elasticsearch?
Code no longer builds due to deprecation of Counter.Set()
When building this code with the latest Prometheus client library, I get:
# github.com/ewr/elasticsearch_exporter
./elasticsearch_exporter.go:300: e.counterVecs["jvm_gc_collection_seconds_count"].WithLabelValues(allStats.ClusterName, stats.Host, collector).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:301: e.counterVecs["jvm_gc_collection_seconds_sum"].WithLabelValues(allStats.ClusterName, stats.Host, collector).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:312: e.counterVecs["thread_pool_completed_count"].WithLabelValues(allStats.ClusterName, stats.Host, pool).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:313: e.counterVecs["thread_pool_rejected_count"].WithLabelValues(allStats.ClusterName, stats.Host, pool).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:330: e.counters["indices_fielddata_evictions"].WithLabelValues(allStats.ClusterName, stats.Host).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:333: e.counters["indices_filter_cache_evictions"].WithLabelValues(allStats.ClusterName, stats.Host).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:336: e.counters["indices_query_cache_evictions"].WithLabelValues(allStats.ClusterName, stats.Host).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:339: e.counters["indices_request_cache_evictions"].WithLabelValues(allStats.ClusterName, stats.Host).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:348: e.counters["indices_store_throttle_time_ms_total"].WithLabelValues(allStats.ClusterName, stats.Host).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:350: e.counters["indices_flush_total"].WithLabelValues(allStats.ClusterName, stats.Host).Set undefined (type prometheus.Counter has no field or method Set)
./elasticsearch_exporter.go:350: too many errors
Turns out Counter.Set() has been removed. the code needs to be rewritten to no longer use Counter, but NewConstMetric() instead.
Inability to differentiate between red and yellow.
According to this ticket:
There is now way to detect if a cluster is red or yellow.
For dashboards like Grafana we need to show the proper color.
It looks like older versions of the plugin used 1,2,3 because the dashboards that reference this plugin on Grafana.com no longer work.
We don't mind publishing our dashboards for Prom 2.0 and the latest elasticsearch_exporter but we have no way to differentiate between yellow and red.
Feature Request: Compatible Grafana Dashboard
I'd like to see a compatible grafana dashboard for this.
Tracking ES node types
Hi guys,
I have a feature request: for elasticsearch_cluster_health_number_of_nodes metric to be split into several dimensions by esrole - i.e. master, client, data, etc.
This way it would be easy to track amount of nodes per type, because, e.g. failure of the client node has a different priority from failure of a data node.
What do you think?
Measure ElasticSearch API timeouts / Check elasticsearch_up
In some cases the elasticsearch_up metric didn't work as expected.
We need to make sure this essential metric is working as expected.
Create example.rules and add an overview of all available metrics
We should provide example recording rules and alerts.
Also an overview of all metric names available in the README should be helpful when evaluating this exporter.
https://github.com/prometheus/node_exporter/blob/master/example.rules
https://github.com/prometheus/mysqld_exporter/blob/master/example.rules
filter cache was already renamed in Elasticsearch 2.0
According to the offical documentation, filter cache was already renamed in 2.0
https://www.elastic.co/guide/en/elasticsearch/reference/2.0/filter-cache.html
The filter cache has been renamed Node Query Cache
In 0.90, it seems had Node Filter Cache and Index Filter Cache.
https://www.elastic.co/guide/en/elasticsearch/reference/0.90/index-modules-cache.html
I confirmed GET /_nodes/stats with 5.5.1 and I got result the following.
In fact, There are no metrics the name of Filter Cache.
"query_cache": {
"memory_size_in_bytes": 14116087,
"total_count": 3772326,
"hit_count": 3001713,
"miss_count": 770613,
"cache_size": 10870,
"cache_count": 15374,
"evictions": 4504
},
"fielddata": {
"memory_size_in_bytes": 37104,
"evictions": 0
},
"request_cache": {
"memory_size_in_bytes": 158617846,
"evictions": 298639,
"hit_count": 61328,
"miss_count": 475752
},
I think that it is better to remove the following, what do you think?
- elasticsearch_indices_filter_cache_evictions
- elasticsearch_indices_filter_cache_memory_size_bytes
If you agree, I'll fix and PR it.
In addition, I would like to add the following.
- elasticsearch_indices_query_cache_hit_rate (hit_count / total_count * 100)
- elasticsearch_indices_query_cache_total_count
- elasticsearch_indices_query_cache_hit_count
- elasticsearch_indices_query_cache_miss_count
- elasticsearch_indices_query_cache_cache_size
- elasticsearch_indices_query_cache_cache_count
- elasticsearch_indices_request_cache_hit_count
- elasticsearch_indices_request_cache_miss_count
Anyway to add custom condition count to metrics
Hi, Is there any way to add custom metrics which count special query result such as:
es_query_count{app="xxx",level="warn",node="node1",...} 10
so I can get warning level log quantity at one hour
Stale cluster health status exposed when `elasticsearch_cluster_health_up` is 0.
I've run into a case where the ES node being scraped by elasticsearch_exporter went down, but elasticsearch_cluster_health_status{color="green"} reported 1, even though elasticsearch_cluster_health_up reported 0.
I think what happens is that the last value of the last successful scrape is retained and re-exposed. This seems wrong to me. I expect none of the metrics other than up, total_scrapes and json_parse_failures to be exposed for failed scrapes.
I didn't verify the nodes and indices collectors in detail, but the code pattern looks the same, so I suspect they have the same issue.
Expose more metrics about segments, search type, ...
Thanks for the great exporter! We are migrating from graphite to prometheus and using this exporter to expose elasticsearch metrics. There are a few metrics we used to collect that don't seem to be exposed:
- elasticsearch.indices.segments.terms_memory_in_bytes
- elasticsearch.indices.segments.index_writer_memory_in_bytes
- elasticsearch.indices.completion.size_in_bytes
- elasticsearch.indices.segments.norms_memory_in_bytes
- elasticsearch.indices.segments.stored_fields_memory_in_bytes
- elasticsearch.indices.segments.doc_values_memory_in_bytes
- elasticsearch.indices.segments.fixed_bit_set_memory_in_bytes
- elasticsearch.indices.search.groups.prefix.query_total
- elasticsearch.indices.search.groups.prefix.query_time_in_millis
- elasticsearch.indices.suggest.total
- elasticsearch.indices.suggest.time_in_millis
- elasticsearch.indices.search.groups.full_text.query_total
- elasticsearch.indices.search.groups.full_text.query_time_in_millis
- elasticsearch.indices.search.groups.more_like.query_total
- elasticsearch.indices.search.groups.more_like.query_time_in_millis
All those metrics are available through the node stat API. Our current implementation based on diamond and is available here.
Any chance you could add those as well?
Thanks!
ElasticSearch 2.0
There are quite a few changes in ElasticSearch 2.0, one of which breaks the JSON deserialization.
What kind of plan would you take for supporting 1.x/2.x?
Collect es.uri from ENV Variable.
Can we have an option of getting the es.uri from env.
The preference should be in the order
- command line flag
- env variable
- default/locahost
add documentation for the new metrics
Unassigned Shards count not showing
When we do the following curl command we see a difference in unassigned shard count and a difference between number_of_nodes vs number_of_data_nodes in prometheus. in prometheus it show unassigned_shards = 0
Nodes = 75
Data_Nodes = 75
$ curl http://x.x.x.x.x.x:9200/_cluster/health | jq .
{
"cluster_name": "logsearch",
"status": "yellow",
"timed_out": false,
"number_of_nodes": 104,
"number_of_data_nodes": 75,
"active_primary_shards": 561,
"active_shards": 1120,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 2,
"delayed_unassigned_shards": 0,
"number_of_pending_tasks": 0,
"number_of_in_flight_fetch": 0,
"task_max_waiting_in_queue_millis": 0,
"active_shards_percent_as_number": 99.8217468805704
}```
timestamp for log entries
Hi,
it would be nice to add timestamps to each log entry to get an idea, when the connection is failing. If you run the exporter in docker/kubernetes you have no idea, when the last connection was broken.
I have no time to dd this feature, but maybe you can added.
Thanks a lot and regards,
Olaf
What are the values for elasticsearch_cluster_health_status?
I don't even see it defined the code...
[v1.0.2.rc2] ES role shows the same on all nodes
Just upgraded to the latest RC, however I noticed that all nodes have almost the same values for the ES role labels, only semi-correct ones are ingest node (though they show that they are also client node, which they are not):
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="true",es_master_node="true",host="elasticin-02",instance="elasticin-02",job="elasticsearch-exporter",name="elasticin-02"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="true",es_master_node="true",host="elasticin-01",instance="elasticin-01",job="elasticsearch-exporter",name="elasticin-01"}
The rest are all wrong:
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticmaster-03",instance="elasticmaster-03",job="elasticsearch-exporter",name="elasticmaster-03"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticmaster-02",instance="elasticmaster-02",job="elasticsearch-exporter",name="elasticmaster-02"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticmaster-01",instance="elasticmaster-01",job="elasticsearch-exporter",name="elasticmaster-01"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticdata-06",instance="elasticdata-06",job="elasticsearch-exporter",name="elasticdata-06"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticdata-05",instance="elasticdata-05",job="elasticsearch-exporter",name="elasticdata-05"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticdata-04",instance="elasticdata-04",job="elasticsearch-exporter",name="elasticdata-04"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticdata-03",instance="elasticdata-03",job="elasticsearch-exporter",name="elasticdata-03"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticdata-02",instance="elasticdata-02",job="elasticsearch-exporter",name="elasticdata-02"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticdata-01",instance="elasticdata-01",job="elasticsearch-exporter",name="elasticdata-01"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticcli-02",instance="elasticcli-02",job="elasticsearch-exporter",name="elasticcli-02"}
elasticsearch_indices_flush_time_seconds{cluster="EFKcluster",es_client_node="true",es_data_node="true",es_ingest_node="false",es_master_node="true",host="elasticcli-01",instance="elasticcli-01",job="elasticsearch-exporter",name="elasticcli-01"}
I double checked our configuration and the roles are defined correctly.
We are using Elasticsearch 5.5.0
Any ideas? any addtional info you need?
Allow Comma separated list of multiple ES servers
Hi there,
Thanks for this great exporter. I have been struggling to get the Prometheus exporter which supports older version i mean < 2.3.1. This works great.
Currently functionality only allows single elastic server instance -es.uri=http:/serverA:9200.
It would be great if we can pass multiple server endpoints like
-es.uri=http:/serverA:9200, serverB:9200,serverC:9200,
This way, even if Server A goes down, exporter still able to collect the data from the second server.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.