mesaguy/ansible-prometheus

Ansible role for the management of Prometheus software and Prometheus exporters

ansible-prometheus's Issues

Unsupported parameter for "command" module

The Find {{ prometheus_software_name_version }} archive contents task fails with the following error:

FAILED! => {
  "changed": false,
  "msg": "Unsupported parameters for (ansible.legacy.command) module: warn. Supported parameters include: _uses_shell, strip_empty_ends, executable, _raw_params, removes, stdin_add_newline, stdin, argv, creates, chdir."}

The unsupported parameter is warn here:

ansible-prometheus/tasks/_install_from_binary.yml

Lines 155 to 159 in c704425

	- name: Find {{ prometheus_software_name_version }} archive contents
	command: 'tar -tf {{ prometheus_software_archive_file }}'
	args:
	# Do not warn that we are running the tar command directly
	warn: false

nginx_exporter_nginxinc is broken because of kebab-case variable names

The nginx_exporter_nginxinc exporter is broken because it declares it's software name as nginx-prometheus-exporter_nginxinc. When overriding the default variables the resulting variable names (eg prometheus_nginx-prometheus-exporter_nginxinc_host) can not be processed by ansible resulting in

ERROR! Invalid variable name in vars specified for IncludeRole: 'prometheus_nginx-prometheus-exporter_nginxinc_host' is not a valid variable name

• ansible version: 2.10.9
• mesaguy.prometheus version: v0.12.38

Enterprise Linux 9 support

Hello already some time Enterprise Linux 9 is available, it would be nice to support it.

v0.13.0 broken at vars/software/mysqld_exporter.yml

mysqld_exporter v0.13.0 will fallback to building from source despite a binary being available as the data in vars/software/mysqld_exporter.yml is bad.

---[snip]---

• name: v0.13.0
  commit:
  sha: ad2847c7fa67b9debafccd5a08bacb12fc9031f1
  timestamp: '2021-05-21T14:14:17'
  timestamp: '2021-05-21T14:15:04'
  prerelease: false
  files: []
  ---[end]---

This looks autogenerated, so not sending a pull request manually fixing it.

prometheus_extra_opts: [] Does not appear to be functioning correctly.

Hey mesaguy,

Once again thanks for your work on this module it is truly appreciated.

Think I have found a bug. It seems that anything being defined in the prometheus_extra_opts: [] array is summarily ignored during deploy. There is no error the options are just not set in the prometheus.service file when the build finishes. For reference, I am trying to pass the following storage options:

prometheus_extra_opts:
  -  '--storage.tsdb.wal-compression'
  -  '--storage.tsdb.retention.time=31d'
  -  '--storage.tsdb.retention.size=20GB'

I am curious if something changed with the last release? I feel like your module used to default to having the the storage.tsdb.retention flag set to a default of 90days if no other options were defined in the vars but that does not seem to be the case anymore.

Thank you in advance for taking a look at this.

Support for rules.yml file

Hey mesaguy,

Thank you for all of your hard work on this code!

Maybe I missed it but I do not see any support for defining a rules.yml file under /etc/prometheus. If i missed it can you point me in the right direction? If I am right, can I make a feature request to support deploying a rules.yml file. I think it could be done similarly to how the prometheus server config file is deployed.

Thanks.

Publishing the role as ansible collection

Dear Mesaguy,

Thank you for the fantastic ansible role!

Have you thought about publishing the role as a collection?

We are thinking about developing a thin abstraction level over your role, which would generate prometheus configs from ansible inventories, and thinking about distribution options. The new "collection" feature of ansible/galaxy looks like a better option than raw roles. It for example allows to share several separate roles as a package, and also allows to share playbooks. Both features look useful in our case. Unfortunately, collections can only use other collections but not roles as dependencies.

Looking at the structure of your role, it seems that you can also benefit from using collections, by being able to split all the software specific roles into separate entities. Your abstraction model with _install.yaml etc, as I understand, can also be transparently translated to a separate roles model.

We will definitely go with your current role, and it is in no way a blocker for us, so this is rather a feature request for the future. Thanks again for your work!

postfix_exporter_kumina exporter

Thanks for the amazing job you're doing on this project, I find it extremely useful and invaluable.

This is more of a question than an actual issue.

I would like to use the postfix_exporter_kumina exporter for some mailservers, but I see its not mentioned in the documentation, and in a wip directory. Does this mean it is currently a "work in progress".

Will it still work if I include it as:

prometheus_components:
  - node_exporter
  - postfix_exporter_kumina

Document adding new exporters and adding exporter metadata

Add documentation detailing the exact steps necessary to add an exporter and the methods used to generate exporter metadata. This issue was brought up in #11

BUG sss_check_problems

Hello :

Line 85 : echo sss_check_problems 1

Replace by : echo sssd_check_problems 1

:)

cloudwatch_exporter does not start in case custom metrics are provided

ansible-prometheus/templates/cloudwatch_exporter.yml.j2

Line 7 in 4c22d37

{{ prometheus_cloudwatch_exporter_cfg | to_nice_yaml(width=80, indent=2) | indent(2) }}

Cloudwatch_exporter cannot be started in case you provide custom metrics - via prometheus_cloudwatch_exporter_cfg.
In that case metrics section is indented to the right which causes error - inability for cloudwatch_exporter to start.

Config file is generated like this:

---
#
# Do not edit manually, this file is managed using automation tools
#
modules: 
  metrics:
  - aws_dimensions:
    - LoadBalancer
    aws_metric_name: ActiveConnectionCount
    aws_namespace: AWS/ApplicationELB

but it should be like this (no indentation for metric section):

---
#
# Do not edit manually, this file is managed using automation tools
#
modules: 
metrics:
- aws_dimensions:
  - LoadBalancer
  aws_metric_name: ActiveConnectionCount
  aws_namespace: AWS/ApplicationELB

See more here:
https://github.com/prometheus/cloudwatch_exporter

@mesaguy

statsd_exporter: no way to specify global defaults

There is no way to specify global defaults ( https://github.com/prometheus/statsd_exporter#global-defaults ) with prometheus_statsd_exporter_cfg.
We could remove mappings: from https://github.com/mesaguy/ansible-prometheus/blob/master/templates/statsd_exporter.yml.j2 but that would not be backward compatible (although this could be fixed in the template by transforming cfg variable). Or we could add prometheus_statsd_exporter_cfg_defaults to the template.
I'm ready to make a PR if it's needed, please advice what way to choose.

By the way https://github.com/mesaguy/ansible-prometheus/blob/master/docs/statsd_exporter.md doesn't mention that we also need to add extra_opts to use templated config.

prometheus_statsd_exporter_extra_opts:
- "--statsd.mapping-config={{prometheus_etc_dir}}/statsd_exporter.yml"

Instead we see confusing: If no configuration content is defined, a default configuration file is utilized.

when my input path and patterns_dir set ok. grok_exporter_fstab ansible error.

when my input path and patterns_dir set ok. grok_exporter_fstab ansible error, but service grok_exporter_fstab staus is running.

---
- name: install prometheus grok_exporter
  hosts: prometheusserver
  vars:
    prometheus_link_etc: false
    prometheus_software_src_dir_suffix: grok_exporter
    prometheus_grok_exporter_fstab_cfg: '{{ lookup("file", "./files/grok_exporter_fstab/grok_exporter_fstab.yml") }}'
    prometheus_components:
     - grok_exporter_fstab

  roles:
    - mesaguy.prometheus

---
global:
    config_version: 2
input:
    type: file
    path: /opt/prometheus/exporters/grok_exporter_fstab/active/example/example.log
    readall: true
grok:
    #patterns_dir: ./logstash-patterns-core/patterns
    patterns_dir: /opt/prometheus/exporters/grok_exporter_fstab/active/patterns
metrics:
    - type: counter
      name: grok_example_lines_total
      help: Counter metric example with labels.
      match: '%{DATE} %{TIME} %{USER:user} %{NUMBER}'
      labels:
          user: '{{.user}}'
server:
    port: 9144

fatal: [host1]: FAILED! => {"msg": "The conditional check 'prometheus_servers' failed. The error was: error while evaluating conditional (prometheus_servers): 'prometheus_servers' is undefined\n\nThe error appears to have been in '/root/devops/roles/mesaguy.prometheus/tasks/_service.yml': line 48, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n - name: Set {{ prometheus_software_name_version }} tgroup facts\n ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n with_items:\n - {{ foo }}\n\nShould be written as:\n\n with_items:\n - "{{ foo }}"\n"}

Incorrect usage of FirewallD module

Hi,

The task _setup_firewall.yml is currently trying to set a port and a source when calling the FirewallD module. However, this is not supported and erroring out. For example:
failed: [hostname.network] (item=10.0.0.1) => { "ansible_loop_var": "prometheus_server_ip", "changed": false, "invocation": { "module_args": { "icmp_block": null, "icmp_block_inversion": null, "immediate": true, "interface": null, "masquerade": null, "offline": null, "permanent": true, "port": "9100/tcp", "rich_rule": null, "service": null, "source": "10.0.0.1", "state": "enabled", "timeout": 0, "zone": "public" } }, "msg": "can only operate on port, service, rich_rule, masquerade, icmp_block, icmp_block_inversion, interface or source at once", "prometheus_server_ip": "10.0.0.1" }

Timing issue on deploy.

Hey mesaguy,

Thanks for the recent update for implementing the rules directory.

I am running in to an issue which I believe is a timing issue. When I try to build with the rules_file: parameter defined in my prometheus.yml file it fails trying to validate the file. I believe this is a result of the rules directory and files not being present when it attempts to validate.

Section in prometheus.yml

rule_files:
 #- /etc/prometheus/rules/rules.yml
 - /opt/prometheus/etc/rules/

Note: neither of the paths above seem to pass validation.

Here is the error:

TASK [mesaguy.prometheus : Setup prometheus-v2.16.0 configuration file] ********
fatal: [server.example.com]: FAILED! => {"changed": false, "checksum": "8311456b0852d76683984c4434af49d5a5e16419", "exit_status": 1, "msg": "failed to validate", "stderr": "  FAILED: \"/opt/prometheus/etc/rules/\" does not point to an existing file\n", "stderr_lines": ["  FAILED: \"/opt/prometheus/etc/rules/\" does not point to an existing file"], 

Thanks in advance for your help.

Support Percona MongoDB Exporter

Hi,
Thanks for all your hard work could it be possible to add support for the Percona MongoDB Exporter.
https://github.com/percona/mongodb_exporter#percona-mongodb-exporter

prometheus_node_exporter_textfiles_directory undefined

Hi

Version: v0.12.22

Hitting an issue with a variable being undefined.

The error was: 'prometheus_node_exporter_textfiles_directory' is undefined
The error appears to be in '../roles/mesaguy.prometheus/tasks/main.yml': line 2, column 3,

Docs describe the variables purpose, but there is no default set for it.

    prometheus_node_exporter_textfiles_directory: '/opt/prometheus/etc/node_exporter_textfiles'

Maybe this needs to be added to defaults?

Using local rules file

I have a custom config file for Prometheus server in which 'rule_files' points to a file on that server. This rule file has to be copied from the controller. In prometheus.yml, the prometheus server config file is created before copying the local rule files:

- name: Setup {{ prometheus_software_name_version }} configuration file
  become: true
  template:
    src: 'templates/generic-config.yml.j2'
    dest: '{{ prometheus_etc_dir }}/prometheus.yml'
    owner: root
    group: '{{ prometheus_group }}'
    mode: 0640
    backup: true
    validate: '{{ prometheus_software_install_dir }}/promtool check config %s'
  notify:
    - Reload Prometheus service

- name: Include task to setup Prometheus rules
  include_tasks: _rules.yml
  when: prometheus_manage_rules | default(false) | bool

This runs into a validation error because the rule file doesn't exist yet:

fatal: [prometheus]: FAILED! => {"changed": false, "checksum": "de45e00cd186d6c11d37e4ce5eebe6fe1d00eb27", "exit_status": 1, "msg": "failed to validate", "stderr": "  FAILED: \"/apps/shared/monitoring/prometheus/rules.yml\" does not point to an existing file\n", "stderr_lines": ["  FAILED: \"/apps/shared/monitoring/prometheus/rules.yml\" does not point to an existing file"], "stdout": "Checking /root/.ansible/tmp/ansible-tmp-1632221852.5072598-70971-236443730807695/source\n\n", "stdout_lines": ["Checking /root/.ansible/tmp/ansible-tmp-1632221852.5072598-70971-236443730807695/source", ""]}

It works fine when I switch the order of these 2 tasks.

Playbook:

- connection: docker
  hosts: prometheus
  vars:
    prometheus_components:
      - prometheus
    prometheus_version: v2.27.1
    prometheus_root_dir: /apps/shared/monitoring/prometheus
    prometheus_manage_rules: true
    prometheus_rules_dir: '{{ prometheus_root_dir }}'
    prometheus_rules_source_dirs:
      - './rules'
    prometheus_server_cfg: '{{ lookup("file", "./prometheus-config.yml") | from_yaml }}'
  roles:
    - mesaguy.prometheus

Am I missing something?

sysvinit issue on Devuan

I think your ansible role is pretty impressive.

Was wondering about you adding support for Devuan. Which is basically
Debian using sysvinit.

All worked so far except it fails here:

TASK [mesaguy.prometheus : Include task to symlink latest node_exporter-v1.0.0 directory to an "active" directory] ****************************************************************************
included: /home/ds/src/ds-santanas-ansible/roles_external/mesaguy.prometheus/tasks/_install_active_symlink.yml for stone

TASK [mesaguy.prometheus : Link /opt/prometheus/exporters/node_exporter/v1.0.0 to /opt/prometheus/exporters/node_exporter/active] *************************************************************
changed: [stone]

TASK [mesaguy.prometheus : Include task to setup capabilites for node_exporter-v1.0.0 files] **************************************************************************************************
skipping: [stone]

TASK [mesaguy.prometheus : Include task to setup node_exporter-v1.0.0 service] ****************************************************************************************************************
included: /home/ds/src/ds-santanas-ansible/roles_external/mesaguy.prometheus/tasks/_service.yml for stone

TASK [mesaguy.prometheus : Verify node_exporter-v1.0.0 permissions] ***************************************************************************************************************************
ok: [stone]

TASK [mesaguy.prometheus : Create environment file for node_exporter service] *****************************************************************************************************************
skipping: [stone]

TASK [mesaguy.prometheus : Include task to setup node_exporter-v1.0.0 sysvinit service] *******************************************************************************************************
fatal: [stone]: FAILED! => {"reason": "Could not find or access '/home/ds/src/ds-santanas-ansible/playbooks/_service_mgr_sysvinit.yml' on the Ansible Controller."}

Installing v0.9.0 nginx_exporter_nginxinc binaries fails

In version 0.9.0 nginx changed the file name schema from ...linux-amd64.tar.gz to ...linux_amd64.tar.gz.

I guess the easiest solution would be to remove all binaries prior to v0.9.0 from vars/software/nginx-prometheus-exporter_nginxinc.yml and switch the architecture to the new schema.

I can send a corresponding PR if you like.