Just a wrapper for helping me test ceph-ansible.
- On hypervisor make VM:
ansible-playbook -i inventory.yml prep.yml
- On VM run ceph-ansible
cd ceph-ansible
ansible-playbook -i inventory.yml site-container.yml.sample
The (initial podman changes have merged) so it is no longer necessary to do this hack:
yum install docker -y
pushd /usr/bin/
mv docker docker.old
ln -s podman docker
popd
Originally from the hack I was able to get to this:
[root@overcloud1 ~]# podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/ceph/daemon latest c99cb4f46f67 27 minutes ago 674MB
[root@overcloud1 ~]# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ffa7fd684e36 docker.io/ceph/daemon:latest /entrypoint.sh About a minute ago Up About a minute ago ceph-mon-overcloud1
[root@overcloud1 ~]#
Inputs:
- fedora28
- docker not installed
- podman installed
container_binary: podman
Results:
- ceph-ansible installs docker
- get a working ceph-mon container but with docker
- playbook fails as it cannot start ceph-mgr contianer
You get the same results however if you use docker in place of podman, so I think I am hitting fedora28 issues.
Inputs:
- fedora29atomic
- docker and podman preinstalled
container_binary: podman
Results
- get a working ceph-mon container with podman (not docker, yay!)
- playbook fails on
create ceph mgr keyring(s)
task
podman
[root@f29atomic ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
[root@f29atomic ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
[root@f29atomic ~]# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a25cae51769c docker.io/ceph/daemon:latest /opt/ceph-container... 6 minutes ago Up 6 minutes ago ceph-mon-f29atomic
[root@f29atomic ~]# podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/ceph/daemon latest 89af895e7d5b 3 days ago 674MB
[root@f29atomic ~]#
Playbook output
TASK [ceph-mon : create ceph mgr keyring(s)] *************************************************************
Monday 03 December 2018 17:46:38 -0500 (0:00:01.314) 0:00:53.841 *******
failed: [192.168.122.241] (item=192.168.122.241) => {"changed": true, "cmd": ["podman", "run", "--rm", "--net=host", "-v", "/etc/ceph:/etc/ceph:z", "-v", "/var/lib/ceph/:/var/lib/ceph/:z", "-v", "/var/log/ceph/:/var/log/ceph/:z", "--entrypoint=ceph", "docker.io/ceph/daemon:latest", "-n", "client.admin", "-k", "/etc/ceph/ceph.client.admin.keyring", "--cluster", "ceph", "auth", "import", "-i", "/etc/ceph//ceph.mgr.f29atomic.keyring"], "delta": "0:00:06.234163", "end": "2018-12-03 22:46:44.954082", "item": "192.168.122.241", "msg": "non-zero return code", "rc": 1, "start": "2018-12-03 22:46:38.719919", "stderr": "2018-12-03 22:46:44.852 7f43dfe2c700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring: (2) No such file or directory\n2018-12-03 22:46:44.855 7f43dfe2c700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication\n[errno 95] error connecting to the cluster", "stderr_lines": ["2018-12-03 22:46:44.852 7f43dfe2c700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring: (2) No such file or directory", "2018-12-03 22:46:44.855 7f43dfe2c700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication", "[errno 95] error connecting to the cluster"], "stdout": "", "stdout_lines": []}
failed: [192.168.122.241] (item=192.168.122.241) => {"changed": true, "cmd": ["podman", "run", "--rm", "--net=host", "-v", "/etc/ceph:/etc/ceph:z", "-v", "/var/lib/ceph/:/var/lib/ceph/:z", "-v", "/var/log/ceph/:/var/log/ceph/:z", "--entrypoint=ceph", "docker.io/ceph/daemon:latest", "-n", "client.admin", "-k", "/etc/ceph/ceph.client.admin.keyring", "--cluster", "ceph", "auth", "import", "-i", "/etc/ceph//ceph.mgr.f29atomic.keyring"], "delta": "0:00:06.212102", "end": "2018-12-03 22:46:51.504912", "item": "192.168.122.241", "msg": "non-zero return code", "rc": 1, "start": "2018-12-03 22:46:45.292810", "stderr": "2018-12-03 22:46:51.409 7f29db70e700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring: (2) No such file or directory\n2018-12-03 22:46:51.413 7f29db70e700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication\n[errno 95] error connecting to the cluster", "stderr_lines": ["2018-12-03 22:46:51.409 7f29db70e700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring: (2) No such file or directory", "2018-12-03 22:46:51.413 7f29db70e700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication", "[errno 95] error connecting to the cluster"], "stdout": "", "stdout_lines": []}
There's that command again
[root@f29atomic ~]# podman run --rm --net=host -v /etc/ceph:/etc/ceph:z -v /var/lib/ceph/:/var/lib/ceph/:z -v /var/log/ceph/:/var/log/ceph/:z --entrypoint=ceph docker.io/ceph/daemon:latest -n client.admin -k /etc/ceph/ceph.client.admin.keyring --cluster ceph auth import -i /etc/ceph//ceph.mgr.f29atomic.keyring
2018-12-03 22:54:13.111 7feaeae4a700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring: (2) No such file or directory
2018-12-03 22:54:13.114 7feaeae4a700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication
[errno 95] error connecting to the cluster
[root@f29atomic ~]# ls -l /etc/ceph/
total 16
-rw-r--r--. 1 root root 789 Dec 3 22:46 ceph.conf
-r--------. 1 ceph ceph 140 Dec 3 22:46 ceph.mgr.f29atomic.keyring
-r--------. 1 root root 77 Dec 3 22:46 ceph.mon.keyring
-rw-r--r--. 1 root root 92 Nov 26 03:18 rbdmap
[root@f29atomic ~]#
Good to see podman progress, I'm more worried about getting it working on f28 than then f29atomic issues I'm seeing.
Next steps:
- Try again with v3.2.1-stable-3.2-luminous-centos-7-x86_64
- Update params based on ceph-ansible CI