crbs/zfsmon

Keep tabs on your ZFS storage with this lightweight script and webapp.

zfsmon

zfs monitor (a catchier name may be forthcoming) is a combination of a Python script that runs on your storage servers as a cron job and a Sinatra-based webapp that runs elsewhere to monitor your ZFS storage.

A picture is worth a thousand words, so here are 5000 of them.

Installing

Storage server (client)

1. Download the egg here.
2. Make sure you have Python >= 2.6 and the requests library installed.
3. easy_install zfsmond-0.3.1-py2.7.egg
4. Copy zfsmond.example.conf to /etc/zfsmond.conf and edit as needed (you will definitely need to set your zfsmon server's hostname)
5. Add /usr/bin/updater.py to the crontab of someone who can run zfs list and zpool list. Every 15 minutes is recommended.

zfsmon server

1. Unzip the webapp tarball
2. bundle install
3. Add a file called auth.yml to this directory with username-password combos for anyone you want to have access to protected resources (deleting hosts, etc.) For example,

        rainbowdash: cool123
   
        someguy: changeme
   

4. ruby start.rb start
5. This will start a server on port 4567 using a sqlite database in the directory you unzipped to.
6. Next, add a reverse proxy from nginx or Apache to the Thin server on 4567. With Apache, you can usually just add these lines to your httpd.conf or to the Vhost you want to host the webapp on:

   ProxyPass           / http://localhost:4567/
   
   ProxyPassReverse    / http://localhost:4567/
   
   ProxyVia            On
   

zfsmon's Issues

Sizes of pools are incorrect

bytes available size is always the same as the used size.
Bytes are base 2^10 not 10^3, i.e 1024 bytes is 1 MB, 1MB is not 1000 bytes

paginate results for datasets and snapshots

Pages time out when you get large numbers of datasets returned, The same is most likely true of snapshots. Modify the erbs to use a limit on the sql and add in buttons to fetch the remaining pages.

Prune pools and datasets when they are no longer in updates

Optimize snapshot updating to avoid sending duplicate data

Sending snapshot data takes a long time, so checking if the same snapshots are already on the monitor server before sending them would speed things up by a lot.

Get vdev information for zpools

Scrape zdb and zpool status for vdev hierarchy and information.

Chrome eventually crashes when left on main page overnight

The specific observed behavior is this:

When left on the main page overnight, by morning that browser window will be crashed with the usual, "Aw, Snap!" A manual refresh temporarily alleviates the issue.

Environment:

cammonitor2-camera% cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.10
DISTRIB_CODENAME=quantal
DISTRIB_DESCRIPTION="Ubuntu 12.10"
cammonitor2-camera% chromium-browser --version
Chromium 22.0.1229.94 Ubuntu 12.10

Please investigate the refresh method.

I'll leave the browser in its crashed state on cammonitor2 for you to investigate.

In the meantime, I've got it running in Firefox and Chrome on Champagne to collect more data.

Ping hosts every now and then to see if they are alive

Get the actual request IP from when the host is first added, then ping that every few minutes to see if it's alive.

shouldn't this be two separate repos?

One for the python module the other for the web application?