Ansible Playbook Error

[freekers]

Ansible Playbook Error

[Freekers]

Hey there!

Looks like it's missing the docker role, which is needed to deploy Docker on the target machine.
Could you please try running the following command from within the folder you've extracted the playbook?
ansible-galaxy install -r requirements.yml --force

[mvicari]

Thanks for the quick reply! Running ansible-galaxy install -r requirements.yml --force results with output:

- downloading role 'docker', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-docker/archive/3.0.0.tar.gz
- extracting geerlingguy.docker to /home/pirate/.ansible/roles/geerlingguy.docker
- geerlingguy.docker (3.0.0) was installed successfully
- downloading role 'rc_local', owned by oefenweb
- downloading role from https://github.com/Oefenweb/ansible-rc-local/archive/v1.0.40.tar.gz
- extracting oefenweb.rc_local to /home/pirate/.ansible/roles/oefenweb.rc_local
- oefenweb.rc_local (v1.0.40) was installed successfully

Then re-running ansible-playbook playbook.yml --ask-become-pass results in the same error unfortunately.

[Freekers]

Hmm, I'm unable to reproduce this on my machine (the classic developer excuse).

Can you try running ansible-playbook playbook.yml --ask-become-pass -vvv and paste the output here?

Also, have you edit the 'playbook.yml' file itself by any chance?

[mvicari]

I haven't modified the 'playbook.yml' file.

Here's the output for ansible-playbook playbook.yml --ask-become-pass -vvv:

ansible-playbook 2.7.7
  config file = /home/pirate/ansible-adguard/ansible.cfg
  configured module search path = ['/home/pirate/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
Using /home/pirate/ansible-adguard/ansible.cfg as config file
SUDO password:
/home/pirate/ansible-adguard/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/home/pirate/ansible-adguard/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /home/pirate/ansible-adguard/hosts inventory source with ini plugin
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/home/pirate/ansible-adguard/playbook.yml': line 216, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


  - name: Deploy Stack using Docker Compose
    ^ here

[Freekers]

Ah, I think I know what's causing this; you're running this on a Raspberry Pi, which is an ARM based device.
The Docker role by geerlingguy I use in the playbook is not compatible with ARM devices (Source: geerlingguy/ansible-role-docker#221 (comment)), but I think we can still make this work.

Edit the requirements.yml file as follows:

- src: geerlingguy.docker_arm
- src: oefenweb.rc_local

Edit the playbook.yml file as follows (around line 99):

  - name: Install Docker and docker-compose
    include_role:
       name: geerlingguy.docker_arm

Please take care of the indentation.

Now, run the following command again from inside the playbook folder:

ansible-galaxy install -r requirements.yml --force

Finally, try running the playbook again:

ansible-playbook playbook.yml --ask-become-pass

[mvicari]

- downloading role 'docker_arm', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-docker_arm/archive/4.0.0.tar.gz
- extracting geerlingguy.docker_arm to /home/pirate/.ansible/roles/geerlingguy.docker_arm
- geerlingguy.docker_arm (4.0.0) was installed successfully
- changing role oefenweb.rc_local from v1.0.40 to unspecified
- downloading role 'rc_local', owned by oefenweb
- downloading role from https://github.com/Oefenweb/ansible-rc-local/archive/v1.0.40.tar.gz
- extracting oefenweb.rc_local to /home/pirate/.ansible/roles/oefenweb.rc_local
- oefenweb.rc_local (v1.0.40) was installed successfully

But am still getting the error. Didn't realize that ARM wasn't supported!

[Freekers]

No problem, I should add that to the README.

Anyway, I will try to look into this coming weekend. I have a Raspberry Pi somewhere or I might see if I can emulate an ARM device use QEMU.

[Freekers]

Right, so the Raspberry Pi I have is an old, original, Raspberry Pi 1 Model B with 512MB of RAM and I forgot how slow it was. Apart from that, the current Docker ARM role by geerling guy, which I use to deploy Docker on the target system, is not compatible with the old RPi 1 ARM variant (yet), so that didn't get me anywhere. I also tried emulating an ARM device on my Windows machine using QEMU but that's all very experimental and harder than I thought it would be...

Anyway, I created a separate branch for ARM devices. The code in there is untested as I have no way of testing it currently. I wrote it blindly based on documentation, so I hope it does what I intend it to do. Please try that one and let me know what you run into.

[mvicari]

Thanks again for the effort.

I cloned the ARM branch using git clone -b arm https://github.com/Freekers/ansible-adguard.git

Edited the hosts file, then ran ansible-playbook playbook.yml --ask-become-pass which resulted in:

ERROR! the role 'geerlingguy.pip' was not found in /home/pirate/ansible-adguard/roles:/home/pirate/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/pirate/ansible-adguard

The error appears to have been in '/home/pirate/ansible-adguard/playbook.yml': line 12, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - geerlingguy.pip
      ^ here

Next, I tried running ansible-galaxy install -r requirements.yml --force which outputs:

- changing role geerlingguy.docker_arm from 4.0.0 to unspecified
- downloading role 'docker_arm', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-docker_arm/archive/4.0.0.tar.gz
- extracting geerlingguy.docker_arm to /home/pirate/.ansible/roles/geerlingguy.docker_arm
- geerlingguy.docker_arm (4.0.0) was installed successfully
- downloading role 'pip', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-pip/archive/2.0.0.tar.gz
- extracting geerlingguy.pip to /home/pirate/.ansible/roles/geerlingguy.pip
- geerlingguy.pip (2.0.0) was installed successfully
- changing role oefenweb.rc_local from v1.0.40 to unspecified
- downloading role 'rc_local', owned by oefenweb
- downloading role from https://github.com/Oefenweb/ansible-rc-local/archive/v1.0.40.tar.gz
- extracting oefenweb.rc_local to /home/pirate/.ansible/roles/oefenweb.rc_local
- oefenweb.rc_local (v1.0.40) was installed successfully

Full output using ansible-playbook playbook.yml --ask-become-pass -vvv:

ansible-playbook 2.7.7
  config file = /home/pirate/ansible-adguard/ansible.cfg
  configured module search path = ['/home/pirate/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
Using /home/pirate/ansible-adguard/ansible.cfg as config file
SUDO password:
/home/pirate/ansible-adguard/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/home/pirate/ansible-adguard/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /home/pirate/ansible-adguard/hosts inventory source with ini plugin
statically imported: /home/pirate/.ansible/roles/geerlingguy.docker_arm/tasks/docker-1809-shim.yml
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/home/pirate/ansible-adguard/playbook.yml': line 215, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


  - name: Deploy Stack using Docker Compose
    ^ here

[Freekers]

Thanks for your detailed answer :)

I've raised an issue with the maintainer of the docker ARM role (geerlingguy) to check if there's a difference in syntax that I'm overseeing, between his ARM and non-ARM docker role that I use.

Link: geerlingguy/ansible-role-docker_arm#20

[Freekers]

Hey @mvicari

I've implemented the feedback I received in the issue that I raised in the repo of the docker ARM role.
Could you please pull the the ARM branch again and test if the issue is now resolved?

Thanks!

[mvicari]

Hello again - thanks for going so far with this!

Output using ansible-playbook playbook.yml --ask-become-pass -vvv:

ansible-playbook 2.7.7
  config file = /home/pi/ansible-adguard/ansible.cfg
  configured module search path = ['/home/pi/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
Using /home/pi/ansible-adguard/ansible.cfg as config file
SUDO password:
/home/pi/ansible-adguard/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/home/pi/ansible-adguard/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /home/pi/ansible-adguard/hosts inventory source with ini plugin
ERROR! the role 'geerlingguy.pip' was not found in /home/pi/ansible-adguard/roles:/home/pi/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/pi/ansible-adguard

The error appears to have been in '/home/pi/ansible-adguard/playbook.yml': line 14, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - geerlingguy.pip
      ^ here

ansible-galaxy install -r requirements.yml --force output:

ansible-galaxy install -r requirements.yml --force
- downloading role 'docker_arm', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-docker_arm/archive/4.1.0.tar.gz
- extracting geerlingguy.docker_arm to /home/pi/.ansible/roles/geerlingguy.docker_arm
- geerlingguy.docker_arm (4.1.0) was installed successfully
- downloading role 'pip', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-pip/archive/2.0.0.tar.gz
- extracting geerlingguy.pip to /home/pi/.ansible/roles/geerlingguy.pip
- geerlingguy.pip (2.0.0) was installed successfully
- downloading role 'rc_local', owned by oefenweb
- downloading role from https://github.com/Oefenweb/ansible-rc-local/archive/v1.0.40.tar.gz
- extracting oefenweb.rc_local to /home/pi/.ansible/roles/oefenweb.rc_local
- oefenweb.rc_local (v1.0.40) was installed successfully

Reran ansible-playbook playbook.yml --ask-become-pass -vvv:

ansible-playbook playbook.yml --ask-become-pass -vvv
ansible-playbook 2.7.7
  config file = /home/pi/ansible-adguard/ansible.cfg
  configured module search path = ['/home/pi/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
Using /home/pi/ansible-adguard/ansible.cfg as config file
SUDO password:
/home/pi/ansible-adguard/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/home/pi/ansible-adguard/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /home/pi/ansible-adguard/hosts inventory source with ini plugin
statically imported: /home/pi/.ansible/roles/geerlingguy.docker_arm/tasks/docker-1809-shim.yml
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/home/pi/ansible-adguard/playbook.yml': line 217, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


  - name: Deploy Stack using Docker Compose
    ^ here

[Freekers]

Aha! It appears the solution was staring at us all along.

You're using ansible 2.7.7. However, prior to Ansible 2.8, the docker_compose task was called docker_service (see synopsis: https://docs.ansible.com/ansible/latest/collections/community/general/docker_service_module.html). Hence Ansible complains it cannot find any task, as it has a different name. Thanks to tgallacher for the hint!

So, there are two approaches to solve this issue:

1. Update Ansible to it's latest version. You'll have to install Ansible using pip to do so, as the latest version in the Debian package repository is indeed 2.7.7 (which you're already running). See 'Installing Ansible': https://dev.to/project42/starting-with-ansible-in-raspberry-pi-2mhm

2. Rename docker_compose to docker_service on line 218 in playbook.yml
   

   ansible-adguard/playbook.yml

   Line 218 in 99cc631

   docker_compose:

[mvicari]

Right under our noses the whole time... haha!

I went with approach 2 first by renaming docker_compose. Running ansible-playbook playbook.yml --ask-become-pass -vvv started spitting good-looking output right away... until the end.

The full traceback is:
  File "/tmp/ansible_apt_payload_rso_8n5x/__main__.py", line 345, in package_status
    pkg = cache[pkgname]
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 303, in __getitem__
    raise KeyError('The cache has no package named %r' % key)

fatal: [192.168.1.111]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "allow_unauthenticated": false,
            "autoclean": false,
            "autoremove": false,
            "cache_valid_time": 0,
            "deb": null,
            "default_release": null,
            "dpkg_options": "force-confdef,force-confold",
            "force": true,
            "force_apt_get": false,
            "install_recommends": false,
            "name": "docker-ce=5:19.03.12~3-0~raspbian-buster",
            "only_upgrade": false,
            "package": [
                "docker-ce=5:19.03.12~3-0~raspbian-buster"
            ],
            "purge": false,
            "state": "present",
            "update_cache": null,
            "upgrade": null
        }
    },
    "msg": "No package matching 'docker-ce' is available"
}
        to retry, use: --limit @/home/pi/ansible-adguard/playbook.retry

Next, I went ahead with approach 1: updating Ansible. Above, I realize now I also needed to install docker-ce which couldn't be installed from the default repositories.

I found and used the instructions here for Ansible: https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible-on-debian

These instructions for installing docker-ce: https://www.shellhacks.com/raspberry-pi-docker-install-docker-on-raspberry-pi

I started with a clean slate, edited the hosts file, and reran ansible-playbook playbook.yml --ask-become-pass -vvv

After a long, seemingly healthy looking install, I receive:

The full traceback is:
  File "/tmp/ansible_docker_compose_payload_0m045h9v/ansible_docker_compose_payload.zip/ansible/modules/cloud/docker/docker_compose.py", line 674, in __init__
  File "/usr/lib/python3/dist-packages/compose/cli/command.py", line 41, in project_from_options
    compatibility=options.get('--compatibility'),
  File "/usr/lib/python3/dist-packages/compose/cli/command.py", line 113, in get_project
    config_data = config.load(config_details, compatibility)
  File "/usr/lib/python3/dist-packages/compose/config/config.py", line 385, in load
    for config_file in config_details.config_files
  File "/usr/lib/python3/dist-packages/compose/config/config.py", line 385, in <listcomp>
    for config_file in config_details.config_files
  File "/usr/lib/python3/dist-packages/compose/config/config.py", line 552, in process_config_file
    validate_against_config_schema(config_file)
  File "/usr/lib/python3/dist-packages/compose/config/validation.py", line 428, in validate_against_config_schema
    schema = load_jsonschema(config_file)
  File "/usr/lib/python3/dist-packages/compose/config/validation.py", line 472, in load_jsonschema
    .format(config_file.filename, VERSION_EXPLANATION))
fatal: [192.168.1.111]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_version": "auto",
            "build": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "debug": false,
            "definition": null,
            "dependencies": true,
            "docker_host": "unix://var/run/docker.sock",
            "files": null,
            "hostname_check": false,
            "nocache": false,
            "project_name": null,
            "project_src": "/opt/adguard",
            "pull": false,
            "recreate": "smart",
            "remove_images": null,
            "remove_orphans": false,
            "remove_volumes": false,
            "restarted": false,
            "scale": null,
            "services": null,
            "ssl_version": null,
            "state": "present",
            "stopped": false,
            "timeout": 10,
            "tls": false,
            "tls_hostname": "localhost",
            "validate_certs": false
        }
    },
    "msg": "Configuration error - Version in \"/opt/adguard/docker-compose.yml\" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g \"2.2\" or \"3.3\") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.\nFor more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/"
}

If I try to run ansible-playbook playbook.yml --ask-become-pass -vvv again:

ERROR! the role 'geerlingguy.pip' was not found in /home/pi/ansible-adguard/roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/pi/ansible-adguard

The error appears to be in '/home/pi/ansible-adguard/playbook.yml': line 14, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - geerlingguy.pip
      ^ here

---

Currently installed:
docker-compose version

docker-compose version 1.21.0, build unknown
docker-py version: 3.4.1
CPython version: 3.7.3
OpenSSL version: OpenSSL 1.1.1d  10 Sep 2019

sudo docker version

Client: Docker Engine - Community
 Version:           20.10.3
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        48d30b5
 Built:             Fri Jan 29 14:33:57 2021
 OS/Arch:           linux/arm
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.3
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       46229ca
  Built:            Fri Jan 29 14:31:57 2021
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

ansible --version

ansible 2.9.17
  config file = /home/pi/ansible-adguard/ansible.cfg
  configured module search path = [u'/home/pi/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0]

[Freekers]

New errors, which means we're making progress ;)

I see you're running docker-compose 1.21.0, which supports docker-compose files up to 2.4 (https://docs.docker.com/compose/release-notes/#1210). Therefore, please change the following line:

ansible-adguard/files/docker-compose.yml

Line 1 in 99cc631

version: '3.7'

From 3.7 to 2.4

Try re-running the following command before you start the playbook again:

ansible-galaxy install -r requirements.yml --force

That should install the missing geerlingguy.pip role.

Sorry for the hassle, but I still don't have an ARM device to test it myself. A friend of mine ordered a RPi4 for me to borrow, but it was dead on arrival...

[mvicari]

No worries! I enjoy troubleshooting... though should have noticed the older docker-compose version! 🤦🏼‍♂️

---

Started from scratch:

• Installed Docker, Docker-Compose, and Ansible per the websites mentioned previously.
• git clone -b arm https://github.com/Freekers/ansible-adguard.git
• Edited hosts
• ran ansible-galaxy install -r requirements.yml --force
• ran ansible-playbook playbook.yml --ask-become-pass -vvv

Received:

The full traceback is:
  File "/tmp/ansible_docker_compose_payload_xh62u5kh/ansible_docker_compose_payload.zip/ansible/modules/cloud/docker/docker_compose.py", line 791, in cmd_up
  File "/usr/local/lib/python3.7/dist-packages/compose/project.py", line 712, in up
    'Encountered errors while bringing up the project.'
fatal: [127.0.0.1]: FAILED! => {
    "changed": false,
    "errors": [
        "ERROR: for unbound-adguard.littlebrain.one  Cannot start service unbound: driver failed programming external connectivity on endpoint unbound-adguard.littlebrain.one (f57bc434486bc2260f413e37a952554398a12b0f141fb1b5b16e3aab2980e0ce): Error starting userland proxy: listen udp4 0.0.0.0:5353: bind: address already in use"
    ],
    "invocation": {
        "module_args": {
            "api_version": "auto",
            "build": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "debug": false,
            "definition": null,
            "dependencies": true,
            "docker_host": "unix://var/run/docker.sock",
            "files": null,
            "hostname_check": false,
            "nocache": false,
            "project_name": null,
            "project_src": "/opt/adguard",
            "pull": false,
            "recreate": "smart",
            "remove_images": null,
            "remove_orphans": false,
            "remove_volumes": false,
            "restarted": false,
            "scale": null,
            "services": null,
            "ssl_version": null,
            "state": "present",
            "stopped": false,
            "timeout": 10,
            "tls": false,
            "tls_hostname": "localhost",
            "validate_certs": false
        }
    },
    "module_stderr": "Creating letsencrypt-adguard.littlebrain.one ... \nCreating unbound-adguard.littlebrain.one     ... \nCreating ouroboros-adguard.littlebrain.one   ... \nHost is already in use by another container\nCreating unbound-adguard.littlebrain.one     ... error\n\nERROR: for unbound-adguard.littlebrain.one  Cannot start service unbound: driver failed programming external connectivity on endpoint unbound-adguard.littlebrain.one (f57bc434486bc2260f413e37a952554398a12b0f141fb1b5b16e3aab2980e0ce): Error starting userland proxy: listen udp4 0.0.0.0:5353: bind: address already in use\nCreating letsencrypt-adguard.littlebrain.one ... done\nCreating ouroboros-adguard.littlebrain.one   ... done\n",
    "module_stdout": "latest: Pulling from pyouroboros/ouroboros\nDigest: sha256:cfa29916459fb8c578fce084ce839a0d3bee478b83a21b6b1d10c6b78bc4a372\nStatus: Downloaded newer image for pyouroboros/ouroboros:latest\nlatest: Pulling from linuxserver/swag\nDigest: sha256:fb6e19aac9fe01a54f0f58ef6a359c185f9011ef4b6793509baaa6c86afadc6b\nStatus: Downloaded newer image for linuxserver/swag:latest\nlatest: Pulling from klutchell/unbound\nDigest: sha256:75de218acd874d50fb49aca48d83f0cbbeadffedf08022adc764dc418b8b657f\nStatus: Downloaded newer image for klutchell/unbound:latest\nlatest: Pulling from adguard/adguardhome\nDigest: sha256:b43525d0a84adc04a463b87eb05e4e6e38175f78821b131596d791d4744c6b6d\nStatus: Downloaded newer image for adguard/adguardhome:latest\n",
    "msg": "Error starting project Encountered errors while bringing up the project."
}

PLAY RECAP *************************************************************************************************************
127.0.0.1                  : ok=31   changed=22   unreachable=0    failed=1    skipped=8    rescued=0    ignored=0

[Freekers]

It seems like you already have something running on port 5353:
Error starting userland proxy: listen udp4 0.0.0.0:5353: bind: address already in use
Could you please check what is occupying the port using sudo netstat -tlpn | grep 5353 ?

If it shows docker-prox, check if there's not already a container running on that port using sudo docker ps | grep 5353

[Freekers]

No response/activity for 1 year. Closing issue.