geerlingguy/internet-pi

Raspberry Pi config for all things Internet.

internet-pi's Issues

Unable to completely finish ansible run

Error is as follows:
TASK [Ensure internet-monitoring environment is running.] **************************************************************
fatal: [192.168.1.247]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (Docker SDK for Python: docker above 5.0.0 (Python >= 3.6) or docker before 5.0.0 (Python 2.7) or docker-py (Python 2.6)) on raspberrypi's Python /usr/bin/python. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter, for example via pip install docker (Python >= 3.6) or pip install docker==4.4.4 (Python 2.7) or pip install docker-py (Python 2.6). The error was: cannot import name credentials"}

Target machine has given the following after manual installation of docker python module:
pi@raspberrypi:~ $ pip install docker
....
Successfully installed backports.ssl-match-hostname-3.7.0.1 docker-4.4.4 websocket-client-0.59.0
pi@raspberrypi:~ $ pip3 install docker
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: docker in /usr/local/lib/python3.7/dist-packages (5.0.0)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/lib/python3.7/dist-packages (from docker) (0.59.0)
Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/lib/python3/dist-packages (from docker) (2.21.0)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from websocket-client>=0.32.0->docker) (1.12.0)

Fix linting errors after merging in internet-monitoring repo

In #6 I merged in the internet-monitoring repo.

I'd like to clean up all the YAML files introduced there so they pass the linting action in CI.

Could not match supplied host pattern, ignoring: internet_pi

I am fairly new to running stuff on RaspberryPI and doing more advanced stuff in Linux. So am probably missing something simple. I am getting the following error when running the playbook:

pi@internetpi:~/internet-pi $ ansible-playbook main.yml
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current
version: 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0]. This feature will be removed from ansible-core in version
2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[WARNING]: Could not match supplied host pattern, ignoring: internet_pi

PLAY [internet_pi] *****************************************************************************************************
skipping: no hosts matched

PLAY RECAP *************************************************************************************************************

My inventory.ini is as follows:

#[internet_pi]
192.168.1.58 ansible_user=pi

#Comment out the previous line and
#uncomment this to run inside Rasberry Pi
#127.0.0.1 ansible_connection=local

I am not installing PiHole yet. I just wanted to set up the internet monitoring part. The IP address is set by DHCP, but it has a reservation on the server, so it will get the same IP all the time. Since it not set to static could this be the issue? I am connecting to the RaspberryPI via ssh from a Windows box to configure it.

I apologize if this is not the correct place to ask this type of question.

Display Download speed and latency on a i2c display like SSD1306

Hello there..I tried this project and it works great...but I wanted to display one or two parameters like the download speed and latency on a i2c display like the SSD1306 using the SDA and SCL pins on the pi ..
I've used this repo to display different items on the display with nice animations and other features :

I can use this repo with the internet-pi one..to display the parameters..but I need some help..where is the file and code in which the speedtest-docker functions take place..and can I write some bash scripts or python along with it..any help is appreciated..Thank You

Ensure

Merge internet-monitoring repo into this one

I'm thinking to make things easier to maintain (and not have dashboards that have no business appearing if you don't have them enabled), I could merge my internet-monitoring repo (which is a fork from another project) into this one, since I don't intend to use it separately anyways...

This would also make it easier to do things like template and/or modify config files through config.yml, making it easier to use everything without forking. For example:

• Speed test frequency is hard-coded (#1).
• Grafana admin password is hard-coded (to wonka).
• Custom domains to ping for HTTP test is hard-coded (my site, GitHub, and Google).

Can't run 'ansible-playbook main.yml' on Pi itself

I have done everything else I am

pi@monitor-pihole:~/internet-monitoring $ ansible --version
ansible 2.9.23

but every time I go to run the play book I get the below issues.

First time I tried:

pi@monitor-pihole:~/internet-monitoring $ ansible-playbook main.yml                            

PLAY [internet_pi] ****************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************
The authenticity of host '192.168.2.100 (192.168.2.100)' can't be established.
ECDSA key fingerprint is SHA256:T6yUqbdRb9Q6DYv7vDeH6zMc0M2qU06r8XadjvpuIzU.
Are you sure you want to continue connecting (yes/no)? yes
fatal: [192.168.2.100]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '192.168.2.100' (ECDSA) to the list of known hosts.\r\[email protected]: Permission denied (publickey,password).", "unreachable": true}

PLAY RECAP ************************************************************************************************************************************************
192.168.2.100              : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   

pi@monitor-pihole:~/internet-monitoring $ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.100  netmask 255.255.255.0  broadcast 192.168.2.255`


2nd time I have tried to run it I got the below
`pi@monitor-pihole:~/internet-monitoring $ ansible-playbook main.yml                            

PLAY [internet_pi] ****************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************
fatal: [192.168.2.100]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: [email protected]: Permission denied (publickey,password).", "unreachable": true}

PLAY RECAP ************************************************************************************************************************************************
192.168.2.100              : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0 `  

ERROR! this task 'ansible.builtin.import_tasks' has extra params

I've been trying to run the playbook on an ubuntu machine, but I get this error message when I run the command :

21:24:36 ~/internet-pi
$ ansible-playbook main.yml
ERROR! this task 'ansible.builtin.import_tasks' has extra params, which is only allowed in the following modules: include, command, import_tasks, add_host, win_command, win_shell, include_role, group_by, meta, shell, script, import_role, raw, set_fact, include_tasks, include_vars

The error appears to be in '/home/[username]/internet-pi/main.yml': line 15, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

 handlers:
   - name: Include handlers.
     ^ here

Is there a way to fix this ?

P.S.: This is the first time I'm using Ansible; while I tried to look on the usual websites for solutions, I wasn't able to find any...

Allow speedtest scrape_interval to be configured

One thing I noticed was after a week or so, the speedtest Docker container (part of the https://github.com/geerlingguy/internet-monitoring project) used something like 2.2 TB of download bandwidth. That's... not gonna fly for someone who has a metered connection.

And it's probably not the best idea for me to be running through that much bandwidth all the time either!

I would like to make it configurable, with a default of maybe 60 minutes.

Add a couple screenshots (Pi-hole and Internet connection dashboard) to README

The README is kinda barren right now. Would look a lot nicer with some pretty screenshots.

Add ability to enable or disable certain services like Pi-hole

Many people including me are already running a homeserver with a adblocking dns such as Pi-hole, Adguard Home and pfblockerng and I would suggest to make a different branch for the monitoring only.
You make awesome videos and thanks a lot.

Add kismet for wireless environment

Hello,

Really nice build! I will try it this evening.

Is it possible to also add kismet for wireless environment scanning?

Thank you!

Don't include shelly-plug and starlink dashboards and prometheus config if they are disabled

If someone sets:

shelly_plug_enable: false
starlink_enable: false

Then the prometheus scraper configs and the Grafana dashboards are still added in, since I haven't appropriately templated those sections of the files yet.

This issue will be open until I get that set up.

It doesn't break anything to have these extra configs present, but it does mean Prometheus sits there trying to hit nonexistent targets forever :)

Allow configuration of default Grafana admin password

Right now it's admin/wonka, though that's not even obvious unless you look it up in https://github.com/geerlingguy/internet-monitoring/blob/master/grafana/config.monitoring#L1

So I'd like to make it configurable. The easiest way would be to add a variable for it after #6 is done.

Nested internet-monitoring directories

Running as found created ~/internet-monitoring/internet-monitoring/ then fails on moving docker-compose into internet-monitoring/grafana/ as it's not there.

Running on Ansible 2.11.2 to Ubuntu 20.04 client.

Adding a trailing / to the src in the file task in internet-monitoring.yml fixed for me.

ansible-playbook error

Hello, I am trying to get this running on a fresh raspberry pi 4 install. everything runs alright until I get to ansible-playbook main.yml
Could anyone help?
I keep getting this error:
TASK [Copy templated internet-monitoring files into place.] ************************************************************failed: [127.0.0.1] (item={'src': 'grafana-config.monitoring.j2', 'dest': 'grafana/config.monitoring'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "779c852d9fb7f4210dbcb7ebeeeaa9b91f625fbf", "item": {"dest": "grafana/config.monitoring", "src": "grafana-config.monitoring.j2"}, "msg": "Destination directory /home/pi/internet-monitoring/grafana does not exist"}

Average parameters are inaccurate as data when pi is shutdown shows 0

Hello there..I have this problem..the average speed over given period is inaccurate as during this particular period i shutdown my pi for a few hours..when I turned it back on..it reported the parameters as 0 during this shutdown period..and now when the average speed is calculated its pretty inaccurate..so is there any way to stop this..or can I delete the data from the shutdown period?
any help is appreciated..Thank You

Use Grafana's new 'State timeline' visualization in Grafana 8.0 for uptime

See: https://grafana.com/docs/grafana/latest/panels/visualizations/state-timeline/

Right now the dashboard uses a 3rd party plugin, but it has some issues like geerlingguy/internet-monitoring#8 — maybe this 1st-party graph type will resolve that issue?

Ensure internet-monitoring ... is running fails due to ansible using python2

Thanks for the dope playbook. Was just about to set up pi-hole, and saw you reference this :)

On a fresh rasbian install, the playbook would fail at the step mentioned in the title, as it couldn't find the python docker sdk (docker on pypi). This was because it was using python2 for some reason. Adding

[internet_pi:vars]
ansible_python_interpreter=/usr/bin/python3

to the inventory seems to have fixed it, but I don't know if this should be necessary?

I'm using ansible ansible 2.9.18 with python version 3.9.2 in fedora remix WSL2 image.

Permission denied - TASK [Ensure internet-monitoring environment is running.]

Playbook fails at the following task :

TASK [Ensure internet-monitoring environment is running.] ********************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Error connecting: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))"}

I tried to give all permissions to the folder with sudo chmod -R a+rwx ~/internet-monitoring/, but unfortunately this did not work.

Unable to install ansible-galaxy

Hi All,

I get the following when I try to install the ansible-galaxy step

pi@raspberrypi:/internet-pi $ dir
ansible.cfg images main.yml tasks
example.config.yml internet-monitoring README.md templates
example.inventory.ini LICENSE requirements.yml
pi@raspberrypi:/internet-pi $ sudo ansible-galaxy collection install -r requirements.yml
sudo: ansible-galaxy: command not found

can you change the favicon

im wondering if i can change the favicon

Issue running playbook

I followed the instructions on this in the readme but still having challenges running the playbook.

pi@mastercon:/internet-pi $ ansible --version
ansible 2.9.6
pi@mastercon:/internet-pi $ python3 --version
Python 3.7.3

ERROR! this task 'ansible.builtin.import_tasks' has extra params, which is only allowed in the following modules: include_vars, import_role, include, include_role, import_tasks, win_shell, meta, raw, add_host, group_by, set_fact, win_command, script, shell, command, include_tasks

The error appears to be in '/home/pi/internet-pi/main.yml': line 15, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

handlers:
- name: Include handlers.
^ here

Pihole does not complete its set up

TASK [Ensure Pi-hole is running.] **************************************************************************************
fatal: [192.168.1.247]: FAILED! => {"changed": false, "errors": [], "module_stderr": "Creating pihole ... \nHost is already in use by another container\nCreating pihole ... error\n", "module_stdout": "latest: Pulling from pihole/pihole\nDigest: sha256:d2f764c4531515a1b93d5a8d1d82272ddc961e580f19c84ed764e4fa863823b1\nStatus: Downloaded newer image for pihole/pihole:latest\n", "msg": "Error starting project Encountered errors while bringing up the project."}

Not sure what to make of it as the system is able to run the other containers just fine.

Add Starlink monitoring

See related:

• geerlingguy/internet-monitoring#6
• danopstech/starlink_exporter#1

Install Error: TASK [Ensure {{ ansible_user }} user is added to the docker group

I received this error message when working through the install.
I looked at the list of users on the system and the user 'ansible_user' does not exist.

Error:

TASK [Ensure {{ ansible_user }} user is added to the docker group.] ****************************************************
fatal: [127.0.0.1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible_user' is undefined\n\nThe error appears to be in '/home/pi/internet-pi/tasks/docker.yml': line 44, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Ensure {{ ansible_user }} user is added to the docker group.\n  ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"}

Full install output:

pi@pi3b2:~/internet-pi $ ansible-playbook main.yml
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current
version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]. This feature will be removed from ansible-core in version
2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

PLAY [internet_pi] *****************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [127.0.0.1]

TASK [Load configuration.] *********************************************************************************************
ok: [127.0.0.1]

TASK [Ensure apt cache is up to date.] *********************************************************************************
ok: [127.0.0.1]

TASK [Check if Docker is already present.] *****************************************************************************
ok: [127.0.0.1]

TASK [Download Docker install convenience script.] *********************************************************************
skipping: [127.0.0.1]

TASK [Run Docker install convenience script.] **************************************************************************
skipping: [127.0.0.1]

TASK [Ensure Docker is started.] ***************************************************************************************
ok: [127.0.0.1]

TASK [Ensure dependencies are installed.] ******************************************************************************
ok: [127.0.0.1]

TASK [Install Docker Compose using Pip.] *******************************************************************************
ok: [127.0.0.1]

TASK [Ensure {{ ansible_user }} user is added to the docker group.] ****************************************************
fatal: [127.0.0.1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible_user' is undefined\n\nThe error appears to be in '/home/pi/internet-pi/tasks/docker.yml': line 44, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Ensure {{ ansible_user }} user is added to the docker group.\n  ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"}

PLAY RECAP *************************************************************************************************************
127.0.0.1                  : ok=7    changed=0    unreachable=0    failed=1    skipped=2    rescued=0    ignored=0

Grafana not automatically importing pre made dashboards

I have verified by shelling into the container that the JSON files do exist for the dashboards but they do not show up as options to select. I have also tried to copy the raw JSON into a grafana dashboard and did not get any results from doing so.

Allow Timezone as a variable

I've always used TZ as a variable and it would be useful to those who are outside the us.

Support for other ARM oprerating systems

with other operating systems such as archlinux arm supporting only raspbain is a bit limiting. It should be possible to modify the existing scripts to use os specific commands based on the current env using the when syntax like this

- name: update system
  shell: sudo pacman -Syu
  tags: - update
  when:
   ansible_distribution == "Archlinux"

In case its a bit difficult to get the ansible_distribution var here is a script that produces a csv file for all hosts specified in the inventory.

- hosts: all
  become: no
  vars:
    output_file: os.csv
  tasks:
    - block:
        # For permisison setup.
        - name: get current user
          command: whoami
          register: whoami
          run_once: yes

        - name: clean file
          copy:
            dest: "{{ output_file }}"
            content: "hostname,distribution,version,release"
            owner: "{{ whoami.stdout }}"
          run_once: yes

        - name: fill os information
          lineinfile:
            path: "{{ output_file }}"
            line: "{{ ansible_hostname }},\
              {{ ansible_distribution }},\
              {{ ansible_distribution_version }},\
              {{ ansible_distribution_release }}"
          # Tries to prevent concurrent writes.
          throttle: 1
      delegate_to: localhost

Make it easy to configure which websites are pinged for HTTP checks

can i please have some help changing what websites it monitors i cant figure out how to change it

ansible_user is undefined

I am connecting through localhost. So I have changed the contents of inventory.ini to
[internet_pi] 27.0.0.1 ansible_connection=local

But when running main.yml, I receive the following error

fatal: [27.0.0.1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible_user' is undefined...

I have tried rebooting and logging out/in as referenced here: #22, but I cannot resolve the issue.

Buster backports apt keys don't exist on key servers

pi@raspberrypi:~ $ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 648ACFD622F3D138
Executing: /tmp/apt-key-gpghome.T88LYjxsyw/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138
gpg: keyserver receive failed: Server indicated a failure

pi@raspberrypi:~ $ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 04EE7237B7D453EC
Executing: /tmp/apt-key-gpghome.2kEPOLdeQc/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC
gpg: keyserver receive failed: Server indicated a failure

TASK [Add Buster backports apt key.] ***************************************************************************************************************************************************************
failed: [192.168.1.100] (item=04EE7237B7D453EC) => {"ansible_loop_var": "item", "changed": false, "cmd": "/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC", "item": "04EE7237B7D453EC", "msg": "Error fetching key 04EE7237B7D453EC from keyserver: keyserver.ubuntu.com", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: Server indicated a failure\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: Server indicated a failure"], "stdout": "Executing: /tmp/apt-key-gpghome.cFtBhk3dIE/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.cFtBhk3dIE/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC"]}
failed: [192.168.1.100] (item=648ACFD622F3D138) => {"ansible_loop_var": "item", "changed": false, "cmd": "/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138", "item": "648ACFD622F3D138", "msg": "Error fetching key 648ACFD622F3D138 from keyserver: keyserver.ubuntu.com", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: Server indicated a failure\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: Server indicated a failure"], "stdout": "Executing: /tmp/apt-key-gpghome.k4z9DVJqE2/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.k4z9DVJqE2/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138"]}

Copy templated internet monitoring

TASK [Copy templated internet-monitoring files into place.] ********************
ok: [127.0.0.1] => (item={'src': 'grafana-config.monitoring.j2', 'dest': 'grafana/config.monitoring'})
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible.errors.AnsibleUndefinedVariable: 'monitoring_speedtest_interval' is undefined
failed: [127.0.0.1] (item={'src': 'prometheus.yml.j2', 'dest': 'prometheus/prometheus.yml'}) => {"ansible_loop_var": "item", "changed": false, "item": {"dest": "prometheus/prometheus.yml", "src": "prometheus.yml.j2"}, "msg": "AnsibleUndefinedVariable: 'monitoring_speedtest_interval' is undefined"}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: yaml.representer.RepresenterError: ('cannot represent an object', AnsibleUndefined)failed: [127.0.0.1] (item={'src': 'prometheus-pinghosts.yaml.j2', 'dest': 'prometheus/pinghosts.yaml'}) => {"ansible_loop_var": "item", "changed": false, "item": {"dest": "prometheus/pinghosts.yaml", "src": "prometheus-pinghosts.yaml.j2"}, "msg": "RepresenterError: ('cannot represent an object', AnsibleUndefined)"}

Allow configuration of speedtest scrape_interval

Right now this would be annoying to change since it's in a different repo, but after #6 is done, I could template that file and the default could be 30m or 60m, and the user could change with monitoring_speedtest_interval: 6h or something like that.

I know for myself, I will be backing it off to maybe 1x per day after I'm finished doing a comparison between Starlink and my Cable ISP.

Error starting userland proxy: bind: address already in use - TASK [Ensure internet-monitoring environment is running.]

After resolving other issues (thus after having run the playbook a few times), the docker containers have been created and re-running the playbook causes this issue :

TASK [Ensure internet-monitoring environment is running.] *******************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "errors": ["ERROR: for internet-monitoring_speedtest_1  Cannot start service speedtest: driver failed programming external connectivity on endpoint internet-monitoring_speedtest_1 (d8dcfa01510ba4d07f5e5438a9dcc9426fd233fee10549ffe4e4b5b9db716662): Error starting userland proxy: listen tcp 0.0.0.0:9798: bind: address already in use", "ERROR: for internet-monitoring_nodeexp_1  Cannot start service nodeexp: driver failed programming external connectivity on endpoint internet-monitoring_nodeexp_1 (8f9ce9324d12c44b75eed9c566c4fd3749979f6383f6e3379f3e31e88c96a3b2): Error starting userland proxy: listen tcp 0.0.0.0:9100: bind: address already in use"], "module_stderr": "Starting internet-monitoring_speedtest_1 ... 
Starting internet-monitoring_ping_1      ... 
Starting internet-monitoring_nodeexp_1   ... 
Host is already in use by another container
Starting internet-monitoring_speedtest_1 ... error

ERROR: for internet-monitoring_speedtest_1  Cannot start service speedtest: driver failed programming external connectivity on endpoint internet-monitoring_speedtest_1 (d8dcfa01510ba4d07f5e5438a9dcc9426fd233fee10549ffe4e4b5b9db716662): Error starting userland proxy: listen tcp 0.0.0.0:9798: bind: address already in use

Host is already in use by another container
Starting internet-monitoring_nodeexp_1   ... error

ERROR: for internet-monitoring_nodeexp_1  Cannot start service nodeexp: driver failed programming external connectivity on endpoint internet-monitoring_nodeexp_1 (8f9ce9324d12c44b75eed9c566c4fd3749979f6383f6e3379f3e31e88c96a3b2): Error starting userland proxy: listen tcp 0.0.0.0:9100: bind: address already in use\nHost is already in use by another container
Starting internet-monitoring_ping_1      ... error
", "module_stdout": "", "msg": "Error starting project Encountered errors while bringing up the project."}

I've tried to remove the containers/purging the system, but when I do this the system has to re-download all images etc., which causes timeout errors and halts the playbook.

Use 'meta: reset_connection` if Docker pi user group changes

Apparently there's now a reset_connection meta task that can reset the SSH connection.

If deploying this playbook to a Pi over SSH, this would supposedly allow the Docker user group changes to occur and not break the use of other docker modules under the pi user account on the first playbook run, which would be nice.

Right now, the first time you run the playbook, when it comes time to start up the environment, you get:

TASK [Ensure internet-monitoring environment is running.] **************************************************************
fatal: [10.0.100.102]: FAILED! => {"changed": false, "msg": "Error connecting: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))"}

speedtest-exporter v3.2.1 and v3.2.2 unhealthy container status

Hi,

not sure if anyone out there faces the same issue where v3.2.0 works perfectly fine.
While version 3.2.1 and 3.2.2 always resulted in the container being unhealthy. I have try recreating on a new container but still resulted in an unhealthy container.

'ansible_user' is undefined - local deployment

When runningansible-playbook main.yml, the playbook fails at :

TASK [Ensure {{ ansible_user }} user is added to the docker group.] **********************************************************
fatal: [127.0.0.1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible_user' is undefined

The error appears to be in '/home/[username]/internet-pi/tasks/docker.yml': line 44, column 3, but maybe
elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


- name: 
Ensure {{ ansible_user }} user is added to the docker group.
  ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}
      
Should be written as:
      
     with_items:
        - "{{ foo }}"
"}

I've tried to replace {{ ansible_user }} with "{{ ansible_user }}", but it doesn't work any better.

ERROR! couldn't resolve module/action 'ansible.posix.synchronize'

Pi 4, ansible 2.9, raspbian lite, the only thing different from defaults is passwords, time zone, and the websites I am pinging. when I run '$ ansible-playbook main.yml" I get:

ERROR! couldn't resolve module/action 'ansible.posix.synchronize'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/home/pi/internet-pi/tasks/internet-monitoring.yml': line 14, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

• name: Synchronize internet-monitoring directory.
  ^ here

---

Ansible --version outputs:

ansible 2.9.23
config file = /home/pi/internet-pi/ansible.cfg
configured module search path = [u'/home/pi/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0]

Thanks!

Ansible main.yml error

Did a fresh install of Raspberry Pi OS
Installed ansible
cloned internet-pi repo
copied config.yml and inventory.ini from example files
ran: ansible-galaxy collection install -r requirements.yml
ran: ansible-playbook main.yml

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/home/pi/internet-pi/main.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

pre_tasks:
- name: Load configuration.
^ here

Still pretty new at Ansible but I am not sure what I am missing.

resolvconf.conf needs modification so local Docker containers get DNS

Over in geerlingguy/internet-monitoring#12 I was scratching my head over some new DNS issues with monitoring containers not being able to resolve DNS names—it caused Grafana to fail to launch, and all pings (once I figured out a way to get Grafana happy again) to fail.

As it turns out, Docker's built-in resolver at 127.0.0.11 was hitting the wrong backend for some reason since the Pi was getting it's LAN IP in /etc/resolv.conf.

So the fix was to uncomment the line:

name_servers=127.0.0.1

Inside /etc/resolvconf.conf.

It's probably important enough a change for compatibility that I should add a bit to the Ansible playbook to update that line in the file using lineinfile.

"ERROR! no action detected in task." when running main.yml

when I run the main.yml file with ansible, I get this error.

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/home/pi/internet-pi/main.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  pre_tasks:
    - name: Load configuration.
      ^ here

Also i take it once installed I can change the frequency of speed tests in the grafina page right?

Create upgrade / update / backup guide

Since it may be non-obvious for those new to Docker.

Basically, the easiest way is to go into each directory in the home folder (in this example, Pi-hole), and pull the images using docker-compose, then restart so it's running the latest image:

# Pi-hole update example
cd ~/pi-hole
docker-compose pull  # pulls the latest images inside the compose file
docker-compose up -d --no-deps  # restarts necessary containers with newer images
docker system prune --all  # deletes unused container images

You would need to do that manually inside each of the associated docker-compose-containing directories from time to time.

It might be good to do the following so this can be somewhat automated:

1. Create a separate playbook or task file that backs up all the relevant configurations/Docker volumes (most notably, pi-hole).
2. Create a playbook that upgrades all images (doing the above, but in each managed directory).
3. Add documentation on how to run the playbook (and recover from backup in case of failure).

Maybe don't do the system prune by default in case someone does need to quickly recover back to an older image with a backup restore.

Excluding the internet monitoring Pi from Pi-hole

One thing that's slightly annoying is every one of the HTTP requests made by the monitoring dashboard is routed through Pi-hole, and because of that, my Pi-hole dashboard basically fills up with this kind of graph:

Most of the requests are coming from 170.20.0.1, which is the monitoring app running in the background in Docker on the same Pi.

Typically, you could just override the default network DNS servers supplied by the router and bypass Pi-hole that way. But it's not quite as easy/fun to do that for one particular Docker container of many on the same instance.

It seems this might be doable using Group Management in Pi-hole 5.

PermissionError on "Ensure internet-monitoring environment is running." task

Hi all,
great idea to combine monitoring and the pi-hole. I tried to install (local) but run in this error:

(fresh installed raspi4 with raspios-buster, python3 set in the inventory)

Ensure internet-monitoring environment is running.
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Error connecting: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Keine Berechtigung'))"}

Do you have an idea whats wrong ?

Integrate internet-pi with Home Assistant?

Hi there..Is it possible to integrate this internet-pi project with Home Assistant..
I tried and am using this project currently on a copy of Raspberry Pi OS...but normally I use my pi to run a Home Assistant server which I use for various purposes..since that also runs a docker, I think it might be easy to integrate this with Home Assistant..maybe as a community add-on?

Ansible failing to work on fresh Pi install

Issue..

Running ansible-galaxy collection install -r requirements.yml returns no response...

Then running ansible-playbook main.yml returns:

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/home/pi/Downloads/internet-pi-master/main.yml': line 6, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  pre_tasks:
    - name: Load configuration.
      ^ here

I don't believe the requirements are being installed

`pi@raspberrypi:/internet-pi $ sudo ansible-galaxy install -r ./requirements.yml -vvv
ansible-galaxy 2.9.23
config file = /home/pi/internet-pi/ansible.cfg
configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible-galaxy
python version = 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0]
Using /home/pi/internet-pi/ansible.cfg as config file
Reading requirement file at './requirements.yml'
pi@raspberrypi:/internet-pi $ ansible-playbook main.yml
ERROR! couldn't resolve module/action 'community.docker.docker_compose'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/home/pi/internet-pi/tasks/internet-monitoring.yml': line 25, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

to the docker group, so this task may fail.

• name: Ensure internet-monitoring environment is running.
  ^ here
  `

Can I use a different user

Usually, when I set up a Raspberry Pi the first thing I do is change the user and password (for security reasons) but when I did it this time I got the following error: FAILED! => {"msg": "Missing sudo password"}
I change back the username and password to the default ones and now it worked so it makes me wonder
1.- Now that is working, if I change again the username and password, would it still work?
2.- How can I directly set it up on a different user?

Installing as user other than pi causes connction aborted issue

I installed this repo as a different user, not the default pi. I got the error below:

Ensure internet-monitoring environment is running.
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Error connecting: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Keine Berechtigung'))"}

Manually added the user to the docker group. Logged out and back in, then re-ran the playbook resolved the issue.