appdynamics/elasticsearch-monitoring-extension

AppDynamics Monitoring Extension for use with Elasticsearch

AppDynamics Monitoring Extension for use with Elasticsearch

This extension requires a Java Machine Agent

Use Case

Elasticsearch is a distributed RESTful search server based on Lucene which provides a distributed multitenant-capable full text search engine. This extension collects metrics using cat endpoints of Elasticsearch REST API and presents them in AppDynamics Metric Browser

Prerequisites

Before the extension is installed, the prerequisites mentioned here need to be met. Please do not proceed with the extension installation if the specified prerequisites are not met.

Elasticsearch's HTTP module should be enabled since this extension collects metrics using cat API over HTTP.

Installation

1. Run 'mvn clean install' from "ElasticsearchMonitorRepo"
2. Unzip the contents of "ElasticsearchMonitor-VERSION.zip" from targets as "ElasticsearchMonitor" and copy the "ElasticsearchMonitor" directory to <MachineAgentHome>/monitors/
3. Configure the extension by referring to the below section.
4. Configure the path to the config.yml file by editing the task-arguments in the monitor.xml file.

       <task-arguments>
           <argument name="config-file" is-required="true" default-value="monitors/ElasticsearchMonitor/config.yml" />
       </task-arguments>
   

5. Restart the machine agent.

Please place the extension in the "monitors" directory of your Machine Agent installation directory. Do not place the extension in the "extensions" directory of your Machine Agent installation directory.

Configuration

Note : Please make sure not to use tab (\t) while editing yaml files. You can validate the yaml file using a yaml validator

Configure the extension by editing the config.yml file in <MachineAgentHome>/monitors/ElasticsearchMonitor/. The metricPrefix of the extension has to be configured as specified here. Please make sure that the right metricPrefix is chosen based on your machine agent deployment, otherwise this could lead to metrics not being visible in the controller.

config.yml

1. Configure the Elasticsearch server in the servers section. This is done by configuring displayName(required), uri(required), username, password or encryptedPassword. The uri only consists of the http:\\{host}:{port}, do not configure the complete cat endpoint here.
2. Next step is setting the catEndpoints. All the servers will share this configuration. To define a cat API you have to provide the following -
   • endpoint - /_cat/{cat api name}. For example to fetch metrics from cat health API, endpoint will be /_cat/health?v. Please refer to elasticsearch documents for API conventions and list of CAT APIs.
   • displayName - Used in metric path, all metrics configured for this endpoint will be listed under this display name in the metric browser.
   • metricPathKeys - if the cat endpoint has any string value in its output then it can be included in the metric path by providing the header of that column. For example, in the below curl the node name(aliased as name) has string values. When this field is configured as metricPathKeys: ["name"] then metrics will be reported as <metricPrefix>|...|N8sjO48|heap.percent for node name N8sjO48

   curl 'http://localhost:9200/_cat/nodes?v&h=heap.percent,ram.percent,cpu,load_1m,name'
   heap.percent ram.percent cpu load_1m name
            49          98   3    0.47 N8sjO48
            53          98   3    0.47 NwL3LHB
   

   • metrics - Configure name and properties. name should match the column header in the response returned by the endpoint. properties include metric qualifier and metrics transforms.

numberOfThreads

Use the following formula for calculating numberOfThreads

numberOfThreads = number of servers *(1 + cat endpoints configured)

for example if there are 7 cat endpoints for one server then numberOfThreads = 1 * (1 + 7) = 8

metricPathReplacements

Please visit this page to get detailed instructions on configuring Metric Path Character sequence replacements in Extensions.

Metrics

Number Statistics exposed through CAT APIs as mentioned here.

Credentials Encryption

Please visit this page to get detailed instructions on password encryption. The steps in this document will guide you through the whole process.

Extensions Workbench

Workbench is an inbuilt feature provided with each extension in order to assist you to fine tune the extension setup before you actually deploy it on the controller. Please review the following document for how to use the Extensions WorkBench

Troubleshooting

Please follow the steps listed in the extensions troubleshooting document in order to troubleshoot your issue. These are a set of common issues that customers might have faced during the installation of the extension.

To troubleshoot this extension please follow these steps -

1. Ensure that elasticsearch is up and running by executing this command -

   curl -v "http://<elasticsearch-host>:<port>"
   

   This command will return a JSON response and the tagline will read You Know, for Search
2. If there is a problem in fetching metrics from one of the configured endpoints then run the command -

   curl -v "http://<elasticsearch-host>:<port>/_cat/<endpoint-configured>?v"
   

   Ensure that the metric names configured in config.yml matches the names in the header of the response. For example, if there are issues in fetching metrics from cat health api then run the command and ensure that you get a valid output and that the header names match with the configured metric names (v parameter in the query is to enable verbose elasticsearch output).

   curl -v "http://<elasticsearch-host>:<port>/_cat/health?v"
   

Contributing

Always feel free to fork and contribute any changes directly via GitHub

Version

Name	Version
Extension Version	3.0.1
Product Tested on	6.6.2
Last Update	15/01/2021
Changes list	ChangeLog

Note: While extensions are maintained and supported by customers under the open-source licensing model, they interact with agents and Controllers that are subject to AppDynamics’ maintenance and support policy. Some extensions have been tested with AppDynamics 4.5.13+ artifacts, but you are strongly recommended against using versions that are no longer supported.

elasticsearch-monitoring-extension's Issues

Fields not correctly parsed

Hi,

After trying to add an endpoint for "/cat/nodes?v", I get an error parsing the fields other than *.percent and name. It wants to parse to "long", but some of them are doubles/floats, like the load* fields.

It would be nice if there is some way to specify the field type, to override the default.

Best regards,
Jeroen

https uri instead of http

Hey guys,

There doesn't seem to be any documentation on whether https:\{host}:{port} will work instead of plain http.

If the appropriate cacerts are installed in the java cacert store in the machine agent, will this work ok?

Thanks for your time!