freifunk-saar / ff-node-monitor Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 2.0 739 KB

Monitoring for Freifunk nodes

License: GNU Affero General Public License v3.0

PLpgSQL 0.99% Rust 27.92% CSS 12.45% JavaScript 40.98% Shell 4.49% Handlebars 13.18%

ff-node-monitor's People

Contributors

Stargazers

Watchers

Forkers

rubo77 freifunk-nord

ff-node-monitor's Issues

Better handling of temporary nightly breakage

Sometimes, this will not compile with the latest nightly (see e.g. #5). What's the best way to handle that?

We could have a file somewhere with a known-working nightly version. Somehow when we push to master, CI should test with that version; but when the nightly CI cronjob runs it should use the latest nightly instead.

How to add the outer frame if included as iframe

I included the node-monitor here: https://freifunk.in-kiel.de/knotenalarm.html

That site includes the node-monitor in an iframe. all emails sent are linking to the site just in that iframe: https://freifunk.in-kiel.de/knotenalarm/ , Which misses our outer frame.

Not sure, how to solve this problem.

I would like to open the site in a frame, but that would only be possible with a cookie set in the node-monitor, i guess

thread 'main' panicked at 'app config table has missing or extra value

# RUST_BACKTRACE=1 /var/lib/ff-node-monitor/src/target/release/ff-node-monitor
🔧
    => address: 127.0.0.1
    => port: 8833
    => log: normal
    => workers: 2
    => secret key: provided
    => limits: forms = 32KiB
    => tls: disabled
    => [extra] ff-node-monitor: { secrets = { action_signing_key = "4ba6997a1469927798f1461fa06d0062ef2224c1bbe3f4c265112e151f8dab09", postgres_url = "postgres://ff-node-monitor@/ff-node-monitor" }, ui = { email_from = "[email protected]", instance_name = "Freifunk Kiel" }, urls = { nodes_url = "https://map.freifunk.in-kiel.de/yanic/nodes.json", root_url = "https://localhost" } }
thread 'main' panicked at 'app config table has missing or extra value: Error { inner: ErrorInner { kind: Custom, line: None, col: 0, message: "missing field `root`", key: ["urls"] } }', libcore/result.rs:945:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:511
   5: std::panicking::continue_panic_fmt
             at libstd/panicking.rs:426
   6: rust_begin_unwind
             at libstd/panicking.rs:337
   7: core::panicking::panic_fmt
             at libcore/panicking.rs:92
   8: core::result::unwrap_failed
   9: ff_node_monitor::config::Config::new
  10: ff_node_monitor::config::fairing::{{closure}}
  11: <rocket::fairing::ad_hoc::AdHoc as rocket::fairing::Fairing>::on_attach
  12: rocket::fairing::fairings::Fairings::attach
  13: rocket::rocket::Rocket::attach
  14: ff_node_monitor::main
  15: std::rt::lang_start::{{closure}}
  16: std::panicking::try::do_call
             at libstd/rt.rs:59
             at libstd/panicking.rs:310
  17: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:105
  18: std::rt::lang_start_internal
             at libstd/panicking.rs:289
             at libstd/panic.rs:374
             at libstd/rt.rs:58
  19: main
  20: __libc_start_main
  21: <unknown>

I use this Rocket.toml

[global.ff-node-monitor.ui]
# The name of your Freifunk community.
instance_name = "Freifunk Kiel"
# The sender address of the emails that are sent by ff-node-monitor.  Make sure
# you have a mail service running locally that is set up to deliver emails from
# that address.
email_from = "[email protected]"

[global.ff-node-monitor.urls]
# The root URL where you will be hosting ff-node-monitor.
root_url = "https://localhost"
# The URL to the hopglass nodes.json file.
nodes_url = "https://map.freifunk.in-kiel.de/yanic/nodes.json"

[global.ff-node-monitor.secrets]
# PostgreSQL credentials.  If you followed the instructions in the README, this
# should be correct.
postgres_url = "postgres://ff-node-monitor@/ff-node-monitor"
# Key used to sign data for confirmation emails.  Generate this key with
# `openssl rand -hex 32`.
action_signing_key = "REDACTED"

[global]
# The address and port on which ff-node-monitor will listen.
address = "127.0.0.1"
port = 8833
# Secret key used by Rocket.  Generate this key with `openssl rand -base64 32`.
secret_key = "REDACTED"

Add info after time of alert

Please add information to the Readme, after which time offline you get automatically alerted by email, that your node is offline and how often you get alerted

Upgrade service

I tried to upgrade to the latest changes, but the service didn't change.

In the /opt/ff-node-monitor/src directory, I tried

sudo systemctl stop ff-node-monitor
git pull
sudo -u ff-node-monitor /opt/ff-node-monitor/.cargo/bin/rustup default $(cat rust-version)
sudo systemctl start ff-node-monitor

error Handling

If you don't enter a valid email, there is no error message

Sort nodes in selectbox case insensitive

If there are a few hundred nodes it is easy to jump to the first letter by typing innnn the selectbox. But if some nodes are capitalized some small, the capital letters are at the front at the moment. this is not very handy

Change license to be less restictive

The AGPL considers network use distribution. This means that every visitor of the website is entitled to a copy of the source. This (theoretically) means a big administrative overhead. I'd like to suggest to use the standart GPL (v2) and putting a link to this repository on the user interface. This would serve the same purpose, but wouldn't be legally binding, so the user would have to go to get the source, not us providing it.

cron is not running correctly

On my system the cron looks ok:

# sudo crontab -l -u ff-node-monitor                                                                                                                                    
*/5 * * * *    http://localhost:8833/cron

but it doesn't seem to run as user ff-node-monitor. There is no meshviewer.json updated.

If I run this this command as root it works fine:

curl http://localhost:8833/cron

But this could have set the downloaded file owner to root somehow?

Where do I have to look? Where is the json file downloaded?

Email should not be case sensitive

at the moment if you add your email address with capitals, it is treated as a different email address. The case should not count as differing.

Umlaut kaputt?

ff-node-monitor/templates/index.html.hbs

Line 19 in b4d0ba8

{{config.ui.instance_name}} Knotenüberwachung

An dieser stelle habe ich ein anderes ü als weiter unten.

ich musste dies aendern, damit die email keinen umlautfehler dort anzeigt:

-  {{config.ui.instance_name}} Knoten<C3><BC>berwachung
+  {{config.ui.instance_name}} Knoten<C3>berwachung

Add Screenshots

could you please add some screenshots to the readme, how it would look like?

like this

error after fresh install on Debian 8.11

$ RUST_BACKTRACE=1 /opt/ff-node-monitor/src/target/release/ff-node-monitor
🔧  Configured for development.
    => address: 0.0.0.0
    => port: 8833
    => log: normal
    => workers: 2
    => secret key: provided
    => limits: forms = 32KiB
    => tls: disabled
    => [extra] ff-node-monitor: { secrets = { action_signing_key = "...", postgres_url = "postgres://ff-node-monitor@/ff-node-monitor" }, ui = { email_from = "[email protected]", in
stance_article_dative = "der", instance_name = "Freifunk Nord Knotenüberwachung" }, urls = { nodes = "https://mesh.freifunknord.de/data/v2/nodes.json", root = "https://nord.freifunk.net/
node-monitor", sources = "https://github.com/freifunk-saar/ff-node-monitor" } }
thread 'main' panicked at 'app config table has missing or extra value: Error { inner: ErrorInner { kind: Custom, line: None, col: 0, message: "Odd number of digits", key: ["secrets", "a
ction_signing_key"] } }', libcore/result.rs:983:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:477
   5: std::panicking::continue_panic_fmt
             at libstd/panicking.rs:391
   6: rust_begin_unwind
             at libstd/panicking.rs:326
   7: core::panicking::panic_fmt
             at libcore/panicking.rs:77
   8: core::result::unwrap_failed
   9: ff_node_monitor::config::Config::new
  10: ff_node_monitor::config::fairing::{{closure}}
  11: <rocket::fairing::ad_hoc::AdHoc as rocket::fairing::Fairing>::on_attach
  12: rocket::fairing::fairings::Fairings::attach
  13: rocket::rocket::Rocket::attach
  14: ff_node_monitor::main
  15: std::rt::lang_start::{{closure}}
  16: std::panicking::try::do_call
             at libstd/rt.rs:59
             at libstd/panicking.rs:310
  17: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:103
  18: std::rt::lang_start_internal
             at libstd/panicking.rs:289
             at libstd/panic.rs:392
             at libstd/rt.rs:58
  19: main
  20: __libc_start_main
  21: <unknown>

automatically focus on email field on load

So you can start typing your email right away.

no database after restart

I rebooted the server and now get this error:

panicked at 'failed to connect to db: BadConnection("could not connect to server: No such file or directory\n\tIs the server running locally and accepting\n\tconnections on Unix domain socket \"/var/run/postgresql/.s.PGSQL.5432\"?\n")', libcore/result.rs:945:5

email should not be case sensitive

If you change case in your email it is treated as another monitor email.

This should be changed so there is only one monitor, no matter the car

improve cron design

HTTP requests have a limited lifetime, the mail sending could take longer than the timeout of the curl /cron request which would the be terminated by the http server or at least result in a curl error. The update mechanism should be done inside the running daemon itself on another thread.

add info about downtime to node list and "back online" notification

When you send a message that a node is back online, you could calculate the time since when it was down and add this info in the mail.

(maybe you could also add some more info to the mail, that could be useful.)

Don't send warnings if ALL nodes are offline

We sometimes have problems with our map. Then all nodes are shown as offline. In such case, the monitor shouldn't send out warning emails.

seems trivial, I would add something like

if numnodes <10 then dontsend emails

Where would be the place in the code, I have to look for to add such a change?

remove nodes that are already monitored from selectbox

vagrant-box: Upgrade to Debian buster

I try to get it runniing on debian 10 in the vagrant-box:
https://github.com/rubo77/ff-node-monitor/commits/buster

but it crashes at install of ssmtp, which seems not to be available any more:

https://unix.stackexchange.com/questions/525235/install-ssmtp-in-debian-buster

node-ids with colon are not working

if you enter a node id with colons like 84:16:f9:9b:e4:12 the nodewatch cannot be activated:

 84:16:f9:9b:e4:12 wurde schon von my@mail überwacht, oder es gibt keinen Knoten mit dieser ID.

the colons should be remmmoved on sending and also trimmed, in case there are some space before or after left

Port to mail crate

The lettre crate we are using for email seems unmaintained. Consider moving to the mail crate.

How do I have to configure mail on my server?

I got the service running on my server: https://freifunk.in-kiel.de/knotenalarm/

But email is not working.

How do I Make sure you have a mail service running locally that is set up to deliver emails from that address?

What packages and tools do I have to install on the server? Maybe you can add this to the Vagrant bootstrap file?

Here I described, how my server is setup

store last Name for nodes

ff-node-monitor/templates/list.html.hbs

Line 34 in 9123db3

<i>?</i> ({{this.monitor.id}}): <span class="gone">verschwunden</span>

This is not so informative, could we store the last name, to show which node is lost?

multi site support

was müsste man anpassen um mehrere sites zu versorgen?

Ich würde gerne einen dienst für Freifunk Nord und einen für Kiel zur verfügung stellen auf dem selben server.

Wenn der service läuft, dann müsste man ja eigentlich nur den service kopieren, wobei er eine andere Rocket.toml benutzt.

Test vagrant on Travis

It would be great if we could test vagrant, or at least the vagrant bootstrap file, on Travis. See #14 (comment) for some ideas.

Cc @rubo77

compiling with older nightly

Due to the currently broken Rocket (see #3 (comment)), I used an older nightly:

$ sudo -u ff-node-monitor /var/lib/ff-node-monitor/.cargo/bin/rustup default nightly-2018-06-09

But then I get an this error:

   Compiling email v0.0.19
error[E0277]: the trait bound `<expression::count::CountStar as expression::Expression>::SqlType: sql_types::ops::Add` is not satisfied
  --> /var/lib/ff-node-monitor/.cargo/registry/src/github.com-1ecc6299db9ec823/diesel-1.3.0/src/expression/count.rs:58:39
   |
58 | #[derive(Debug, Clone, Copy, QueryId, DieselNumericOps)]
   |                                       ^^^^^^^^^^^^^^^^ the trait `sql_types::ops::Add` is not implemented for `<expression::count::CountStar as expression::Expression>::SqlType`
   |
   = help: consider adding a `where <expression::count::CountStar as expression::Expression>::SqlType: sql_types::ops::Add` bound
   = help: see issue #48214
   = help: add #![feature(trivial_bounds)] to the crate attributes to enable

relative stylesheet link

The link to the stylesheet should be relative, so it will work if served via more than one path

Add more info to monitors

please add some fields to the monitors dtabase table.

start - the timestamp when the monitor was added
offline_count - each time, the node is reported offline, this goes up, so you can have a statistic over time, how often your node went offline since start
offline_time - add the time, the node was offline when it comes back online, so you have a stats how much time the node was offline

error: component 'rustfmt'

I got this today when upgrading to the latest master commit:

error: component 'rustfmt' for target 'x86_64-unknown-linux-gnu' is unavailable for download for channel nightly-2020-06-20

Rocket failed to launch due to an I/O error. thread 'main' panicked at 'Permission denied (os error 13)

after adding login right to the user into the postgresql database i am stuck here now:

sudo -u ff-node-monitor RUST_BACKTRACE=1 /var/lib/ff-node-monitor/src/target/release/ff-node-monitor       
🔧
    => address: 127.0.0.1
    => port: 80
    => log: normal
    => workers: 2
    => secret key: provided
    => limits: forms = 32KiB
    => tls: disabled
    => [extra] ff-node-monitor: { secrets = { action_signing_key = "xxxxx", postgres_url = "postgres://ff-node-monitor@/ff-node-monitor" }, ui = { email_from = "[email protected]", instance_name = "Freifunk Kiel" }, urls = { nodes = "https://map.freifunk.in-kiel.de/yanic/nodes.json", root = "http://localhost" } }
🛰  Mounting '/':
    => GET /
    => GET /list?<form>
    => GET /list
    => POST /prepare_action
    => GET /run_action?<form>
    => GET /cron
    => GET /static/<file..>
Error: Rocket failed to launch due to an I/O error.
thread 'main' panicked at 'Permission denied (os error 13)', /var/lib/ff-node-monitor/.cargo/registry/src/github.com-1ecc6299db9ec823/rocket-0.3.12/src/error.rs:204:17
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:511
   5: std::panicking::continue_panic_fmt
             at libstd/panicking.rs:426
   6: std::panicking::begin_panic_fmt
             at libstd/panicking.rs:413
   7: <rocket::error::LaunchError as core::ops::drop::Drop>::drop
   8: ff_node_monitor::main
   9: std::rt::lang_start::{{closure}}
  10: std::panicking::try::do_call
             at libstd/rt.rs:59
             at libstd/panicking.rs:310
  11: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:105
  12: std::rt::lang_start_internal
             at libstd/panicking.rs:289
             at libstd/panic.rs:374
             at libstd/rt.rs:58
  13: main
  14: __libc_start_main
  15: <unknown>

I use the nightly version you added and this Rocket.toml:

[global.ff-node-monitor.ui]
# The name of your Freifunk community.
instance_name = "Freifunk Kiel"
# The sender address of the emails that are sent by ff-node-monitor.  Make sure
# you have a mail service running locally that is set up to deliver emails from
# that address.
email_from = "[email protected]"

[global.ff-node-monitor.urls]
# The root URL where you will be hosting ff-node-monitor.
root = "http://localhost"
# The URL to the hopglass nodes.json file.
nodes = "https://map.freifunk.in-kiel.de/yanic/nodes.json"

[global.ff-node-monitor.secrets]
# PostgreSQL credentials.  If you followed the instructions in the README, this
# should be correct.
postgres_url = "postgres://ff-node-monitor@/ff-node-monitor"
# Key used to sign data for confirmation emails.  Generate this key with
# `openssl rand -hex 32`.
action_signing_key = "xxxxx"

[global]
# The address and port on which ff-node-monitor will listen.
address = "127.0.0.1"
port = 80
# Secret key used by Rocket.  Generate this key with `openssl rand -base64 32`.
secret_key = "xxxx/ZqsfLpRSw="

add link to map in email

It would be great to define a custom link for the warning email, so you can click on that link if you get a warning and visit directly the status of the node in your meshviewer

node adding don't work

in Kiel, I can select a node and hit "HInzufügen" now without error, the following page sais, it has added the node to my monitors, but if I check, the node is still unmonitoered. it is not in my overview

cron not working

Here I get an internal server error: https://freifunk.in-kiel.de/knotenalarm/cron

how can I debug this?

in Rocket.toml I have this nodelist configured: https://hopglass.freifunk.in-kiel.de/nodes.json

unresolved import `syntax::parse::common`

services:    Compiling rocket_contrib v0.3.12
services: error[E0432]: unresolved import `syntax::parse::common`
services:  --> /var/lib/ff-node-monitor/.cargo/registry/src/github.com-1ecc6299db9ec823/rocket_codegen-0.3.12/src/utils/parser_ext.rs:5:20                                                                
services:   |                                                                                                                                                                                             
services: 5 | use syntax::parse::common::SeqSep;                                                                                                                                                          
services:   |                    ^^^^^^ Could not find `common` in `parse`                                                                                                                                
services: error: aborting due to previous error

component 'rustfmt' for target 'x86_64-unknown-linux-gnu' is unavailable

# sudo -u ff-node-monitor ~ff-node-monitor/.cargo/bin/rustup default $(cat rust-version)
info: syncing channel updates for 'nightly-2020-06-20-x86_64-unknown-linux-gnu'
info: latest update on 2020-06-20, rust version 1.46.0-nightly (2d8bd9b74 2020-06-19)
error: component 'rustfmt' for target 'x86_64-unknown-linux-gnu' is unavailable for download for channel nightly-2020-06-20
Sometimes not all components are available in any given nightly.

Error when sending mail in vagrant box

Now I get a similar error like #63 when trying to add monitor in the vagrant machine:

Error: Response was a non-`Responder` `Err`: I/O-Error: Connection refused (os error 111).
Warning: This `Responder` implementation has been deprecated.
Warning: In Rocket v0.5, `Result<T, E>` implements `Responder` only if `E` implements `Responder`. For the previous behavior, use `Result<T, Debug<E>>` where `Debug` is `rocket::response::Debug`.
Warning: Responding with 500 Internal Server Error catcher.

I defined the example configuration for msmtp from the Readme and used my googlemail account, which has 2FA disabled

Readme: add info which folders can be deleted after install

there are ~1.5GB of space used during install

please add info which folders can be deleted without problem. Or add instructions how to build locally and deploy only the needed files to a production server

possibility to use external stylesheet

We use a CMS for our Freifunk page. There we could generate a file that could work as template for the ff node motor.

Or do you know a better solution to include the monitor pages into an existing website?

An Iframe would work quite bad, because the monitor pages would come from a different domain or port than the homepage... Hmm ... Maybe if the monitor pages are served in a subfolder..

If that works, please hint to a working solution in the readme

Emails crash the service, maybe missing ssl certificate or wrong mail config?

When trying to add a new node to monitor, I get this error:

Error: Response was a non-`Responder` `Err`: Connecting(Io(Custom { kind: Other, error: Ssl(Error { code: ErrorCode(1), cause: Some(Ssl(ErrorStack([Error { code: 337047686, library: "SSL routines", function: "tls_process_server_certificate", reason: "certificate verify failed", file: "../ssl/statem/statem_clnt.c", line: 1915 }]))) }, X509VerifyResult { code: 64, error: "IP address mismatch" }) })).
Warning: Responding with 500 Internal Server Error catcher.

send node name in email

please send the selected nodename if you selected via the selectbox. It is very confusing to get a mail with just the mac address

No access possible via URL

I try to configure my Rocket.toml with the right URL, so I can access the service in my VM via web. I tried:

root = "http://localhost"
root = "http://10.19.0.2"
root = "http://10.19.0.2/"

but nothing works.

Locally on the shell I can access it via http:/localhost:8080

the VM is accessible, i tried a python minimal server which works fine on port 8000

Can you please elaborate, what I have to think about?

sudo: cargo: command not found

When I come to the step

sudo -u ff-node-monitor cargo build --release

I get the error

sudo: cargo: command not found

I tried on Debian stretch and on jessie plain install.

ERROR: Database connection failure

I get this error after upgrading debian to buster and postgresql to 11:

RUST_BACKTRACE=1 /opt/ff-node-monitor/src/target/release/ff-node-monitor 
🔧 Configured for production.
    => address: 0.0.0.0
    => port: 8000
    => log: critical
    => workers: 14
    => secret key: generated
    => limits: forms = 32KiB
    => keep-alive: 5s
    => tls: disabled
Warning: environment is 'production', but no `secret_key` is configured
Error: Database configuration failure: 'postgres'
Error: A table named `databases` was not found for this configuration
...

Maybe because I recreated the database and something is missing? We got an update from postgresql 9.6 to 11 which created a new folder /var/lib/postgresql/11 but the old database is still inside the folder /var/lib/postgresql/9.6

unknown locale de_DE

in my english vagrant box, i get the error

$ sudo -u postgres psql -c 'CREATE DATABASE "ff-node-monitor" WITH OWNER = "ff-node-monitor" LC_COLLATE = "de_DE.UTF-8 UTF-8" TEMPLATE template0;'
ERROR:  invalid locale name: "de_DE.UTF-8 UTF-8"

Do I have to activate the new locales in postgresql somehow?

it is installed:


# echo $LANG
de_DE.UTF-8

Error: Database configuration failure: 'postgres'

After migrating to a new server, and importing and upgrading Postgres to v11 i get this error:

Error: Database configuration failure: 'postgres'
Jul 02 09:42:27 freifunk ff-node-monitor[22562]: Error: A table named `databases` was not found for this configuration
Jul 02 09:42:27 freifunk ff-node-monitor[22562]: thread 'main' panicked at 'could not connect to DB for migrations', src/libcore/option.rs:1036:5