jonhiggs/flamingzombies

A lightweight monitoring service

flamingzombies's Issues

Order the lists in the man pages

They're unsorted everywhere.

add task to check swap usage

Its too hard to chase the source of errors when using the `fz_error` task.

Work out how to make debugging easier when the tests are distributed across hosts.

AND and OR the gates.

Make the notification gates parameter behave as documented.

Tasks need to include data in the notifications

It's hard to tune the tasks if you don't know by how much they're failing by.

Detect flapping tasks

Add a way to notify when a task is in a flapping state.

Validate that notifiers don't reference unknown gates

Support a config directory (`/etc/flamingzombies.d/`)

All files in the config directory gets concatenated to to the end of the config file.

Log stderr from task scripts

Support muting tasks.

It can be annoying when a task renotifies every hour, but you know you're not going to look at it for a while. You don't want to mute the whole ntfy channel, but you don't need to hear about that task anymore.

Repeat notifications

Add a gate to renotify periodically.

Send the failing gateset to the notifier environment

A flappy monitor could have any status. A hint that the the flap gate tripped is needed.

Support passing env vars to tasks

I've yet to need this, but I expect that sooner or later I will.

Add a directory of example configurations

Demonstrate how it can be used with a set of common and less common examples.

Make the tasks support the 124 exit code

As documented in fz-tasks(7).

Add a deadline to fzctl

it will allow for a more reliable task/fz_errors plugin.

Document flamingzombies.toml

add task to check the age of the file

fails if it gets too old.

don't segfault if a gate's script doesn't exist.

time="2024-05-01T13:40:44+10:00" level=info msg="executing task" file=lib/fz/task.go task_hash=2103319012 task_name=flappy
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x52c115]

goroutine 6 [running]:
github.com/jonhiggs/flamingzombies/lib/fz.Gate.IsOpen({{0xc000014520, 0xc}, {0xc00001c1a0, 0x1a}, {0x0, 0x0, 0x0}}, 0xc0000c6000)
        /home/jon/src/flamingzombies/lib/fz/gate.go:67 +0xa95
github.com/jonhiggs/flamingzombies/lib/fz.ProcessNotifications.func1()
        /home/jon/src/flamingzombies/lib/fz/notification.go:26 +0x10c7
created by github.com/jonhiggs/flamingzombies/lib/fz.ProcessNotifications in goroutine 1
        /home/jon/src/flamingzombies/lib/fz/notification.go:20 +0x1a
exit status 2

Add grace to the timeouts

Adding grace time to the timeouts makes managing the timeout in the tasks easier; you don't have to deduct any time from ${TIMEOUT} to avoid losing to the fz's context deadline.

task to check that directory contains new files

From a directory of backup files, make sure that the newest one is at newer than n seconds.

If the directory doesn't exist, return 3.
If the glob doesn't match anything, return 1.
If none of the matching files are new enough, return 1.

Run as an unprivileged user

installation instructions for debian

task to check that filesystem is mounted

add task to check disk inodes free

`error_count` doesn't increase when error is encounterd

email notifier content

Print more useful information in the email notifications.

Consider chaning the transport between fz and fzctl to HTTP.

A raw tcp port has problems like:

• you can't use your load balancers to route it.
• I suspect it's currently single threaded, and possibly breaking the fz_errors task.

Consider when to reset `LastResultOutput`

I've noticed situations where tasks are failed, but the notification presents a successful output.

support `fzctl help` command

fzctl --help is awkward when you're in a cmd subcmd mode of thinking.

include a bash_completion script for fzctl

installation instructions for alpine linux

Support listening on multiple network interfaces

add task to check disk space free

Fix the initial value for a `task.LastState` to "unknown"

add task to check Eaton UPS

Defer notifications

Add support to defer a notification allowing it some time to auto-resolve before notifying.

Configurable `deadline_grace` and docs

Make the deadline_grace configurable, and ensure that it plus the timeout isn't > the frequency or retry_frequency.

add task to check load average

Allow querying the notifier details with fzctl

This will probably require splitting out tasks and notifier actions with a new subcommand.

Support task decriptions

Jamming everythign into the name is getting difficult.

OpenBSD installation instructions

Add more example configuration files.

fz task plugin

Add a task plugin to detect if an fz daemon has experienced any errors for any of its tasks.

Support configurable `from` address for the email notifier.

This should probably be configured by sending custom environment variables back to the notifier.

add task to measure tls expiration

make sure task names don't contain spaces

makes working with the output of fzctl harder for no need.