mvonthron/munin-influxdb

Munin to InfluxDB/Grafana gateway

munin-influxdb's Issues

fetch config json file mixed up

When munin contains several hosts with common metrics (for instance uptime or cpu), there is only one "tag" section with one host.

"tags": {
    "cpu": {
      "domain": "testnodes",
      "host": "pmtest3",
      "plugin": "cpu"
    },
}

This makes future fetch operation write all values from different hosts with the tag host="pmtest3" instead of the right one.

in fetch.py, the tags are computed out of the config file:

    return [{
            "measurement": measurement,
            "tags": config['tags'][measurement],
            "time": fields['time'],
            "fields": {key: value for key, value in fields.iteritems() if key != 'time'}
        } for measurement, fields in data.iteritems()]

As a dirty workaround, I forced domain and tags config before the return statement:

    for measurement, fields in data.iteritems():
        config['tags'][measurement]['host'] = host
        config['tags'][measurement]['domain'] = domain

cron job missing

Looks like there was no cron job created during installation to refresh the munin data periodically.
Where it should be located at all? Not a serious issue, I created it for myself which does the job.

Error writing ... to InfluxDB: list index out of range

Hi,

i try to export some munin data to influxdb and get the following errors (no data at all in the influx-db):

  ✓ Exported 850 RRD files to XML (/tmp/munin-influxdb/xml)

InfluxDB: Please enter your connection information
  - host/handle [localhost]: fubar
  - database [munin]: 
munin database doesn't exist. Would you want to create it? [y]/n: 
Group multiple fields of the same plugin in the same time series? [y]/n: 

Uploading data to InfluxDB:
  ✓ Connection to database "munin" OK
  Progress: [                                                  ] 0%fubar.diskstats_latency.sdb
[...]
  ✗ Error writing load to InfluxDB: list index out of range
  ✗ Error writing diskstats_utilization.sda to InfluxDB: list index out of range
  ✗ Error writing if_err_SWITCH to InfluxDB: list index out of range
  ✗ Error writing entropy to InfluxDB: list index out of range
  ✗ Error writing fail2ban to InfluxDB: list index out of range
[...]
  ✗ Error writing entropy to InfluxDB: list index out of range
  ⚠ Could not read file for down: 
  ⚠ Could not read file for up:
[...]
✓ Munin data successfully imported to fubar/db/munin
✗ Error: [Errno 2] No such file or directory: '/home/fubar/.config/munin-fetch-config.json'

404 Page not found

Hi
Thanks for this great tool :)

I've got an error 404 when i try to connect to my influxdb server from my munin box
What i'm typing in :

• host/handle [localhost]: myinfluxdbserver.mydomain
• port [8086]: 8086
• user [None]: influxdbuser
• password: influxdbuserpass
  ⚠ Could not connect to database: 404: 404 page not found

Am i doing something wrong ??
Nothing appears in influxdb.log and tcpdump shows nothing too
Makgin a curl -G 'http://myinfluxdbserver.mydomain:8086/query?q=SHOW+USERS&u=influxdbuser&p=influxdbuserpass' throws the good result

Thanks for any advice

Influxdb doesn't update

When I run muninflux fetch, the command executes without problems, but the database on influxdb doesn't update.
Its only update if I run muninflux import again.

What the magic to this works fine?

tks

Fetch cron job does not run with user munin

FETCH_CONFIG file is stored at /root/.config/munin-fetch-config.json once import has been run as root.

The cron job that runs as munin returns no output, can not read the FETCH_CONFIG?

creating grafana dashboard fails

Hi!
first of all: great tool!
Unfortunately it stops when it comes to creating the grafana dashboard:

[...]
Dashboard title [Munin Dashboard]: 
Number of graphs per row [2]: 
Show min/max/current in legend [y]/n: 
Progress: [###########                                       ] 22%✗ Error: invalid literal for int() with base 10: '58982.4'

I'm not that into python to figure out what's wrong but maybe you can help me with that?

Cheers
Fiete

Small update:
After running that again, the error changed to:

Progress: [######                                            ] 12%✗ Error: threshold2

No xml output in /tmp/munin-influxdb/xml/

Dear all,

I have an issue when I launch: muninflux import:

Munin to InfluxDB migration tool

Exploring Munin structure
✗ Could not process datafile (/tmp/munin/rrd/datafile), will read www and RRD cache instead
Reading Munin www cache: (/tmp/munin/html)
Reading Munin RRD cache: (/tmp/munin/rrd)
SERVER1: [##################################################] 100%
✗ coercing to Unicode: need string or buffer, NoneType found

Exporting RRD databases:
Progress: [##################################################] 100%
✓ Exported 1137 RRD files to XML (/tmp/munin-influxdb/xml)

The /tmp/munin-influxdb/xml path is empty...

If I try manually: rrdtool dump /tmp/munin/rrd/SERVER1/SERVER1-redis_memory_subsite-memory-g.rrd test.xml, works!

Installation via pip creates no executable

So i have tried installing munin-influxdb via pip. The installation succeeds, but i have no muninflux executable. Since muninflux is just an alias to munin-influxdb.sh, i started a system-wide search for both files, but i cannot find them. It looks that just the files under /lib are installed via pip.

Am i just using it wrong, or do i really have to install munin-influxdb manually?

can not fetch data for interval smaller than 2 days

Firstly, this is a very good library, works very smooth.

When I see my grafana dashboard, I can only see the stats for the past 2 days. If I select an interval less than 2 days, like I want to see for the past 6 hours, I can't do that.

In the second picture, when I have selected an interval of 6 hours, I can't see anything.
Why so ?

InfluxDB 0.9 compatibility

Hi,

I'm wondering if this tool is compatible with InfluxDB 0.9+ ?

Thanks.

SyntaxError: invalid syntax

Hi,

I do not know much from python and perhaps the following is a stupid error but could you please help me?

After executing:
./muninflux import

Here are the messages:

Traceback (most recent call last):
  File "bin/import.py", line 6, in <module>
    from munininfluxdb import munin
  File "build/bdist.linux-x86_64/egg/munininfluxdb/munin.py", line 6, in <module>
  File "/usr/lib/python2.6/site-packages/munin_influxdb-0.1-py2.6.egg/munininfluxdb/settings.py", line 136
    for f, h, p, field in self.iter_fields()
      ^
SyntaxError: invalid syntax

Do I have something mixed up in python?

Thanks

Handling munin subdomains

Hi,

thanks for this awesome program, It's realy helpfull.

Munin allows to define an arbitrary long subdomain. For instance:
acadis.org;physical;tesla:memory.swap.label swap

This is not handled correctly by the program, resulting in data not exported. As a workaroud, I'll set domain as "acadis.org-physical" to match my needs.
As a matter of facts, this is as if we flattened all the subdomains tree. It works well for finding the statefile too.
Maybe this is acceptable as a general solution ?

use_node_name breaks datafile import

running muninflux import doesn't succeed processing /var/lib/munin/datafile. The offending line reads like example.com;host:use_node_name 0 - opposed to the other, plugin-related lines, there is only one string after the host specification, instead of multiple .-separated ones.

Can't import

Hi there,

I can't figure out why import fails.
It doesn't show an error itself, maybe you can help to find the problem.
Here is my log:

# ./muninflux import
Munin to InfluxDB migration tool
--------------------
Exploring Munin structure
  ✗ Could not process datafile (/var/lib/munin/datafile), will read www and RRD cache instead
Reading Munin www cache: (/var/cache/munin/www)
/usr/local/lib/python2.7/dist-packages/bs4/__init__.py:166: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html5lib"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

To get rid of this warning, change this:

 BeautifulSoup([your markup])

to this:

 BeautifulSoup([your markup], "html5lib")

  markup_type=markup_type))
DO: [##################################################] 100%
fhserve.com: [##################################################] 100%
mh: [##################################################] 100%
Reading Munin RRD cache: (/var/lib/munin)
fhserve.com: [##################################################] 100%
DO: [##################################################] 100%
mh: [##################################################] 100%
  ✗ coercing to Unicode: need string or buffer, NoneType found

Exporting RRD databases:
  Progress: [##################################################] 100%
...
...
...
  Progress: [##############################################################################################################################################################################################################################################################################################] 573%

 ✓ Exported 19086 RRD files to XML (/tmp/munin-influxdb/xml)

InfluxDB: Please enter your connection information
  - host/handle [localhost]: 
/usr/local/lib/python2.7/dist-packages/influxdb-2.7.1-py2.7.egg/influxdb/influxdb08/client.py:551: FutureWarning: get_database_list is deprecated, and will be removed in future versions. Please use ``InfluxDBClient.get_list_database`` instead.
  - port [8086]: 
  - user [None]: root
  - password: 
  - database [munin]: 
/usr/local/lib/python2.7/dist-packages/influxdb-2.7.1-py2.7.egg/influxdb/influxdb08/client.py:194: FutureWarning: switch_db is deprecated, and will be removed in future versions. Please use ``InfluxDBClient.switch_database(database)`` instead.
Group multiple fields of the same plugin in the same time series? [y]/n: y

Uploading data to InfluxDB:
  ✓ Connection to database "munin" OK
✗ Error: 


#

Any ideas?