breadlysm/speedflux

Script to periodically run the Speedtest CLI application by Ookla and post results to InfluxDB.

speedflux's Issues

Error on logs:

I have a error on container:

2022-01-09 01:12:18,697 [INFO] Speedtest Successful...Writing to Influx
2022-01-09 01:12:18,697 [INFO] Speedtest Data:
time: 2022-01-09T01:12:18Z
ping: 25.278ms
download: 44.073784Mb/s
upload: 10.949944Mb/s
isp: XXXXXXXXXXXXXXXXXXXXXXXXXX
ext. IP: XXX.XXX.XXX.XXX
server id: 12746
server location: (SFR @ Paris)

Process Process-11:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/app/speedflux/data.py", line 37, in speedtest
speedflux.INFLUXDB.process_data(data_json)
File "/app/speedflux/influx.py", line 193, in process_data
data = self.format_data(data)
File "/app/speedflux/influx.py", line 118, in format_data
tags = self.tag_selection(data)
File "/app/speedflux/influx.py", line 189, in tag_selection
options[tag] = tag_switch[tag]
KeyError: 'None'
2022-01-09 01:13:48,728 [DEBUG] http://XXX.XXX.XXX.XXX:8086 "POST /write?db=speedtests HTTP/1.1" 204 0
2022-01-09 01:13:48,729 [INFO] Ping data written successfully
2022-01-09 01:13:48,743 [DEBUG] http://XXX.XXX.XXX.XXX:8086 "POST /write?db=speedtests HTTP/1.1" 204 0
2022-01-09 01:13:48,744 [INFO] Ping data written successfully

Please help me?

Add support for Influxdb2

It looks like the write commands and authentication remain largely intact or have compatible methods.

The line protocol data though will need a change to include extra buckets.
https://docs.influxdata.com/influxdb/v2.0/api/#tag/Write

Specify 2 test servers

Hi, not sure how feasible it would be and it's probably quite niche but I'd like to speedtest 2 servers equally.
My use case is that I'd like to easily compare the results from my ISP's Speedtest server and an external Speedtest server as the results are often dramatically different.
Thanks in advance.

No shebang, files in CRLF

main.py has no shebang line, and its in CRLF DOS format.

Fix dockerfile install for speedtest

Speedtest-cli was using BinTray. The service was sunset and the dockerfile is producing an error.

Update dockerfile to use their new service.

Evolution suggestions

Hello,
I've been using speedflux to monitor my internet connection for a while now and I wanted to know if it was possible somehow to get a few evolutions:

• ability to set planning in a cron manner instead of interval.
• I saw that it was possible (at least it used to be possible) with speedtest cli to pass --country ... if the command is not deprecated, would it be possible to implement this option for speedflux? as of now we can either filter with specific servers list or no filter.
  Thanks :)

Step by step install?

This looks to be a great tool, but for the heck of it cannot figure out how to install it.

Does anyone have step-by-step instructions on how to install it and edit files?

"No data" InfluxDB Error: Gateway Timeout

I installed on Synology NAS using docker compose with completely default settings (unchanged yml file), but I'm getting no data on my grafana chart, and error InfluxDB Error: Gateway Timeout hovering over the red tag in the corner.

Does anybody know how to fix this?

make an arm compatible docker image for raspberry pi and other ARM based devices

Refactor config to be more accessible and expandable

should make Config and possibly other functions accessible globally without importing individual modules. This will make adding new config options for influx2 easier.

Speedtest Data writing failed

Hi,
I get this error after starting the image:

speedflux_1  | 2021-08-18 12:02:35,123 [INFO] Manual server choice : ID = None
speedflux_1  | 2021-08-18 12:02:35,123 [INFO] Speedtest Successful...Writing to Influx
speedflux_1  | 2021-08-18 12:02:35,123 [INFO] Speedtest Data:
speedflux_1  |             time: 2021-08-18T12:02:35Z
speedflux_1  |             ping: 26.77ms
speedflux_1  |             download: 62.947632Mb/s
speedflux_1  |             upload: 31.390568Mb/s
....
speedflux_1  |             server location: (PLUTEX GmbH @                 Bremen)
speedflux_1  |
speedflux_1  | Process Process-8:
speedflux_1  | Traceback (most recent call last):
speedflux_1  |   File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
speedflux_1  |     self.run()
speedflux_1  |   File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
speedflux_1  |     self._target(*self._args, **self._kwargs)
speedflux_1  |   File "/app/speedflux/data.py", line 37, in speedtest
speedflux_1  |     speedflux.INFLUXDB.process_data(data_json)
speedflux_1  |   File "/app/speedflux/influx.py", line 193, in process_data
speedflux_1  |     data = self.format_data(data)
speedflux_1  |   File "/app/speedflux/influx.py", line 118, in format_data
speedflux_1  |     tags = self.tag_selection(data)
speedflux_1  |   File "/app/speedflux/influx.py", line 189, in tag_selection
speedflux_1  |     options[tag] = tag_switch[tag]
speedflux_1  | KeyError: 'None'

Running with a influxdb-1.8.9-1

Best Regards

Add new config options for influx2

Add new config options for additional settings needed.

Bucket names, Org Names, Keys.

Allow users to use both or either Influx versions.

Null config sets will not export to that version of influxdb.

Global flag for influx version to use: INFLUX1, INFLUX2, BOTH.
Flag is not user accessible and is decided by input settings.

Max retries exceeded with url: /query?q=SHOW+DATABASES

Hi,

Been using the script with Grafana for the last 6 months without any issue, however, ran into this and can't seem to get passed it.

`During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/admin/speedtest-to-influxdb/./main.py", line 41, in
main()
File "/Users/admin/speedtest-to-influxdb/./main.py", line 12, in main
influx = Influx(config)
File "/Users/admin/speedtest-to-influxdb/speedflux/influx.py", line 14, in init
self.init_db()
File "/Users/admin/speedtest-to-influxdb/speedflux/influx.py", line 17, in init_db
databases = self.client.get_list_database()
File "/usr/local/lib/python3.9/site-packages/influxdb/client.py", line 704, in get_list_database
return list(self.query("SHOW DATABASES").get_points())
File "/usr/local/lib/python3.9/site-packages/influxdb/client.py", line 521, in query
response = self.request(
File "/usr/local/lib/python3.9/site-packages/influxdb/client.py", line 332, in request
response = self._session.request(
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='192.168.1.15', port=8088): Max retries exceeded with url: /query?q=SHOW+DATABASES (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x111041a30>: Failed to establish a new connection: [Errno 61] Connection refused'))`

Updated the script to latest version, same issue

Thanks

Update documentation

Once new feature is completed, documentation will need updated before pushing to master.

Should test locally for a couple of days before pushing changes.

Speedtest Fail - starting yesterday the 21st

been working like a charm - and then its over - happily talking to my influx db - but seems to be unhappy with ookla -- their client change or something?

Newbie - sorry for the silly questions

Hi.... trying to get the basic connectivity between Speedflux and Influx DB... I get the following errors in the Speedflux log.
I have one ID and password configured because InfluxDB forced me to create one through their Web UI. Running dockers on Unraid 6.9.2. Also attaching snapshots of both the InfluxDB config and Speedflux Config. thanks.

("Couldn't initiate InfluxDB <2", InfluxDBClientError('401: {"code":"unauthorized","message":"Unauthorized"}'))

jitter ?

would it be possible to add to the dashboard a ping jitter panel?
here is an exemple of a speed test with a jitter display http://ting.speedtestcustom.com/

Making the script more robust

This is a minor irk of mine while using the docker container on unRAID - the docker dies when speedtests randomly fail and I have to restart again. I can work around this with docker's retry functionality, but would be nice to just work out of the box.

There's two things I think would make this better.

1. Retry when writing to influxDB - Sometimes on a fluke, writing fails, crashing the whole docker. Would just be nice to retry 3 or 5 times and then just spit an error message if it fails (but keep running the tests at every interval). Maybe wait 30s or so between each one? (Unless there's seriously someone doing speedtests every 2 minutes).
2. try/except inside the while loop - Mostly just to have the script continue when something fails randomly (and keep the sleep timer going)

Thanks for maintaining this project, There's several solutions out there, but the one I was using previously was using its own UI, and I wanted something to use grafana to visualize, so this is great!

How to show speeds from different speedtest servers

I have the SPEEDTEST_SERVER_ID environment variable empty, so I assume it will just choose the best server but Grafana looks like this

with the server and location fields empty

Speedtest will not run after initial launch

Sorry to be logging another issue, but it seems the script will not launch speedtest on interval set:

TEST_INTERVAL = int(os.getenv('SPEEDTEST_INTERVAL', '300')) * 60
PING_INTERVAL = int(os.getenv('PING_INTERVAL', '300'))

It will write speedtest to DB on launch, but that's it. 5 minutes later it'll just perform the ping:

2021-05-30 10:38:34,861 [DEBUG] http://127.0.0.1:8086 "POST /write?db=speedtest HTTP/1.1" 204 0
2021-05-30 10:38:34,862 [INFO] Ping data written successfully
2021-05-30 10:38:34,892 [DEBUG] http://127.0.0.1:8086 "POST /write?db=speedtest HTTP/1.1" 204 0
2021-05-30 10:38:34,893 [INFO] Ping data written successfully
2021-05-30 10:38:56,726 [INFO] Speedtest Successful...Writing to Influx
2021-05-30 10:38:56,727 [INFO] Speedtest Data:
time: 2021-05-30T08:38:56Z
ping: 13.684ms
download: 941.8116Mb/s
upload: 599.403496Mb/s
isp: Orange
ext. IP: 86.xxx.xx.xx
2021-05-30 10:38:56,752 [INFO] Speedtest data written successfully
2021-05-30 10:43:35,737 [DEBUG] Starting new HTTP connection (1): 127.0.0.1:8086
2021-05-30 10:43:35,749 [DEBUG] http://127.0.0.1:8086 "POST /write?db=speedtest HTTP/1.1" 204 0
2021-05-30 10:43:35,750 [INFO] Ping data written successfully
2021-05-30 10:43:35,783 [DEBUG] http://127.0.0.1:8086 "POST /write?db=speedtest HTTP/1.1" 204 0
2021-05-30 10:43:35,784 [INFO] Ping data written successfully

NB - I downloaded the newest version of the script, not sure why it would run once and miss its intervals.

Create new functions to export to influx2

Likely will need to refactor Influx1 exporter.

possible solution
Lean functions that handle common tasks in influx class.
Inherit those into separate objects for influx 1 and 2.

another option is just separate connection methods for 1&2 with separate exports under the same object.