haxorof / ansible-role-docker-ce Goto Github PK
View Code? Open in Web Editor NEWAnsible role to install Docker CE on AlmaLinux/Rocky/CentOS/Fedora/RHEL(Redhat)/Ubuntu/Debian/Mint/Raspbian
License: MIT License
Ansible role to install Docker CE on AlmaLinux/Rocky/CentOS/Fedora/RHEL(Redhat)/Ubuntu/Debian/Mint/Raspbian
License: MIT License
Whoops! I had been developing against CentOS this whole time, in preparation for our real servers (RHEL). (In the past, I've had trouble trying to run RHEL locally, due to licensing issues, so I run CentOS to test what I develop for RHEL.) I finally have it to the point where my Ops colleague ran it and...
FAILED! => {"changed": false, "msg": "Distribution RedHat is not supported by this role!"}
I didn't realize that this role didn't explicitly accommodate RedHat.
As far as I know, it would just be another simple synonym for CentOS, and CentOS version numbers mirror RHEL's.
Refactor tasks and split up in more files which fits together etc.
It's a shot in the dark that you might know this (it involves Vagrant), but here goes.
I'm trying to see if I can satisfy the "1.1 Ensure a separate partition for containers has been created" check.
The following configurations are stripped down to remove anything that's not directly related to this question.
The Vagrantfile
happens to use the vagrant-persistent-storage
plugin (which I just discovered today when looking for a solution). Even if you don't know the particulars of that plugin, maybe you could tell me what the end state of the disk configuration would need to be for the role's "dm.directlvm_device=/dev/sdb1"
configuration to use it.
Vagrantfile
:
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.define :slave do |slave|
if Vagrant.has_plugin?("vagrant-persistent-storage")
slave.persistent_storage.enabled = true
slave.persistent_storage.location = "~/development/sourcehdd.vdi"
slave.persistent_storage.size = 5000
slave.persistent_storage.filesystem = 'ext4'
#slave.persistent_storage.partition = false
end
end
slave.vm.hostname = "centos-jenkins-slave"
# Disable the new default behavior introduced in Vagrant 1.7, to
# ensure that all Vagrant machines will use the same SSH key pair.
# See https://github.com/mitchellh/vagrant/issues/5005
slave.ssh.insert_key = false
slave.vm.box = "geerlingguy/centos7"
slave.vm.provision "ansible" do |ansible|
ansible.verbose = "v"
ansible.playbook = "playbook.yml"
end
end
end
playbook.yml
:
roles:
- role: haxorof.docker-ce
docker_enable_audit: true
docker_enable_ce_edge: true
docker_daemon_config:
icc: false
init: true
userns-remap: default
disable-legacy-registry: true
live-restore: true
userland-proxy: false
log-driver: journald
storage-driver: devicemapper
storage-opts:
- "dm.directlvm_device=/dev/sdb1"
Which yields:
$ sudo fdisk -l
Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0009da9f
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 167772159 82836480 8e Linux LVM
Disk /dev/sdb: 5242 MB, 5242880000 bytes, 10240000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdb1 1 10233404 5116702 8e Linux LVM
Disk /dev/mapper/centos-root: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-swap: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-home: 30.1 GB, 30056382464 bytes, 58703872 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/vps-vps: 5234 MB, 5234491392 bytes, 10223616 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
...and:
$ sudo df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/centos-root 52403200 1198136 51205064 3% /
devtmpfs 239084 0 239084 0% /dev
tmpfs 250008 0 250008 0% /dev/shm
tmpfs 250008 4476 245532 2% /run
tmpfs 250008 0 250008 0% /sys/fs/cgroup
/dev/sda1 1038336 156620 881716 16% /boot
/dev/mapper/centos-home 29337604 35892 29301712 1% /home
192.168.56.1:/Users/jjackson/.vagrant.d/cache/geerlingguy/centos7 487260160 280297600 206706560 58% /tmp/vagrant-cache
vagrant 487260160 280553620 206706540 58% /vagrant
tmpfs 50004 0 50004 0% /run/user/1000
Not that I really know what I'm doing with any of the above, but I keep ending up with the following, despite different attempts:
TASK [haxorof.docker-ce : Enable and start Docker service] *********************
fatal: [slave]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to start service docker: Job for docker.service failed because the control process exited with error code. See \"systemctl status docker.service\" and \"journalctl -xe\" for details.\n"}
https://access.redhat.com/solutions/2991041
Red Hat Enterprise Linux Server release 7.4 (Maipo)
(but I assume it would happen on CentOS 7.4, too).
We into this today after an app server became inaccessible. We got the aforementioned error when docker exec
ing into the container (then afterward, we got driver failed programming external connectivity on endpoint
when trying to restart the container--not sure if this bit is related or not).
I performed a shotgun fix by removing all containers, restarting the docker service, and re-running the docker-compose up on the stack.
From the KB article:
Root Cause
- docker on RHEL does not support daemon restart and live-restore.
- docker systemd unit file has 'MountFlag=true enabled. This is not compatible with live-restore.
I know we're setting MountFlags=slave
, but I'm not sure if that's associated with this root cause.
I wonder if the role should prevent live-restore
with certain OS versions or configurations.
Hi,
To create a kubernets clusters is necessary the installation of docker 17.03 which is the last supported.
Trying to change the name of package gives the error:
changed": true, "msg": "Error: Package: docker-ce-17.03.2.ce-1.el7.centos.x86_64 (docker-ce-st
able)\n Requires: docker-ce-selinux >= 17.03.2.ce-1.el7.centos\n Available: docker-ce-selinux-17.03.0.ce-
1.el7.centos.noarch (docker-ce-stable)\n docker-ce-selinux = 17.03.0.ce-1.el7.centos\n
Which says package docker-ce-selinux is incompatible.
To solve the problem it needs install both at the same time:
yum install -y --setopt=obsoletes=0
docker-ce-17.03.0.ce-1.el7.centos
docker-ce-selinux-17.03.0.ce-1.el7.centos
It could be good sometimes to just skip the installation part and just do the configuration and necessary restarts. Also the other way around could be needed during some phases.
Add variable where you can specify docker_users
to be added to the docker group
OS: CentOS Linux release 7.4.1708 (Core)
My ultimate target is RHEL 7.4, but I'm working on CentOS, ATM.
Have you used namespaces yet? I'm having issues.
I'm going to paste in my notes from my own private issue tracker:
Details: https://docs.docker.com/engine/security/userns-remap/#enable-userns-remap-on-the-daemon
Notes:
In trying to implement this, I added the following to the playbook, however docker wouldn't restart.
vars:
...
docker_daemon_config:
...
userns-remap: default
...
/var/log/messages
told me that docker was having a problem using the extant files/filesystem, so I did a vagrant destroy slave
.
Once I ran vagrant up
from scratch, docker started, but I go this:
$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Pull complete
Digest: sha256:97ce6fa4b6cdc0790cda65fe7290b74cfebd9fa0c9b8c38e979330d547d22ce1
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:301: running exec setns process for init caused \"exit status 40\"": unknown.
ERRO[0002] error waiting for container: context canceled
That led me to these references:
I then ran the following and rebooted the VM:
$ sudo grubby --args="user_namespace.enable=1" --update-kernel="$(grubby --default-kernel)"
$ sudo grubby --args="namespace.unpriv_enable=1" --update-kernel="$(grubby --default-kernel)"
Upon restart:
$ docker run --rm hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Pull complete
Digest: sha256:97ce6fa4b6cdc0790cda65fe7290b74cfebd9fa0c9b8c38e979330d547d22ce1
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
Todo:
This is really an Ansible question, but I'm having trouble getting help with it.
Some of my hosts will have a different device for dm.directlvm_device
--not a problem, I can use a variable for that. However, some hosts don't have a dedicated dm.directlvm_device
. I'm not sure how to toggle the presence of the dm.directlvm_device
element (or alternatively, I guess, the storage-opts
key).
Do you have any suggestions?
- hosts: dockerhosts
vars:
docker_compose_version: 1.16.1
docker_enable_audit: true
docker_enable_ce_edge: true
docker_daemon_config:
icc: false
init: true
disable-legacy-registry: true
live-restore: true
userland-proxy: false
log-driver: journald
storage-driver: devicemapper
storage-opts:
- "dm.directlvm_device=/dev/sdb2"
# remove for real servers?
debug: true
force_secrets_recreation: false
do_decryption_test: false
do_dump: false
myname: "{{myuser | default('hosting') }}"
media_share_ignore_mount_errors: false
ignore_user_creation_errors: false
roles:
- role: haxorof.docker-ce
tags: [docker]
- role: franklinkim.docker-compose
tags: [docker-compose]
become: yes
Task below indicated as changed every time.
TASK [haxorof.docker-ce : Update APT cache] **************************************************************************************************************************
changed: [labrat]
I'm new to this role as of today (and just heard of Docker Bench from this project's README). I'm trying this out with the following on a CentOS 7.4 VM:
roles:
- role: haxorof.docker-ce
docker_enable_audit: true
docker_daemon_config:
icc: false
init: true
userns-remap: default
disable-legacy-registry: true
live-restore: true
userland-proxy: false
log-driver: journald
storage-driver: devicemapper
I tried running the docker run
from Docker Bench, but I hit some hurdles. Here's the first:
[root@centos-jenkins-slave vagrant]# docker run -it --net host --pid host --cap-add audit_control \
> -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \
> -v /var/lib:/var/lib \
> -v /var/run/docker.sock:/var/run/docker.sock \
> -v /usr/lib/systemd:/usr/lib/systemd \
> -v /etc:/etc --label docker_bench_security \
> docker/docker-bench-security
docker: Error response from daemon: cannot share the host's network namespace when user namespaces are enabled.
See 'docker run --help'.
How do I run Docker Bench against a host provisioned with this role?
Add initial support for Ubuntu and Debian distributions.
Thanks for the helpful role!
I understand that I use it on a beta (until next week or so?) of Fedora 29, but maybe the issue hits other installations as well:
when I apply the role I get:
TASK [haxorof.docker-ce : Install additional packages (PiP)] *************************************************************************************************
[DEPRECATION WARNING]: Invoking "pip" only once while using a loop via squash_actions is deprecated. Instead of using a loop to supply multiple items and
specifying `name: {{ item }}`, please use `name: [u'{{ _docker_additional_packages_pip }}']` and remove the loop. This feature will be removed in version
2.11. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
failed: [127.0.0.1] (item=[u'docker', u'docker-compose']) => {"changed": false, "cmd": ["/bin/pip2", "install", "docker", "docker-compose"], "item": ["docker", "docker-compose"], "msg": "stdout: Requirement already satisfied: docker in /usr/lib/python2.7/site-packages (3.5.1)\nRequirement already satisfied: docker-compose in /usr/lib/python2.7/site-packages (1.22.0)\nRequirement already satisfied: websocket-client>=0.32.0 in /usr/lib/python2.7/site-packages (from docker) (0.53.0)\nRequirement already satisfied: docker-pycreds>=0.3.0 in /usr/lib/python2.7/site-packages (from docker) (0.3.0)\nRequirement already satisfied: backports.ssl-match-hostname>=3.5; python_version < \"3.5\" in /usr/lib/python2.7/site-packages (from docker) (3.5.0.1)\nRequirement already satisfied: ipaddress>=1.0.16; python_version < \"3.3\" in /usr/lib/python2.7/site-packages (from docker) (1.0.22)\nRequirement already satisfied: six>=1.4.0 in /usr/lib/python2.7/site-packages (from docker) (1.11.0)\nRequirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/lib/python2.7/site-packages (from docker) (2.20.0)\nCollecting PyYAML<4,>=3.10 (from docker-compose)\n Using cached https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz\nRequirement already satisfied: texttable<0.10,>=0.9.0 in /usr/lib/python2.7/site-packages (from docker-compose) (0.9.1)\nRequirement already satisfied: dockerpty<0.5,>=0.4.1 in /usr/lib/python2.7/site-packages (from docker-compose) (0.4.1)\nRequirement already satisfied: enum34<2,>=1.0.4; python_version < \"3.4\" in /usr/lib/python2.7/site-packages (from docker-compose) (1.1.6)\nRequirement already satisfied: docopt<0.7,>=0.6.1 in /usr/lib/python2.7/site-packages (from docker-compose) (0.6.2)\nRequirement already satisfied: jsonschema<3,>=2.5.1 in /usr/lib/python2.7/site-packages (from docker-compose) (2.6.0)\nRequirement already satisfied: cached-property<2,>=1.2.0 in /usr/lib/python2.7/site-packages (from docker-compose) (1.5.1)\nRequirement already satisfied: idna<2.8,>=2.5 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (2.7)\nRequirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (3.0.4)\nRequirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (1.24)\nRequirement already satisfied: certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (2018.10.15)\nRequirement already satisfied: functools32; python_version == \"2.7\" in /usr/lib/python2.7/site-packages (from jsonschema<3,>=2.5.1->docker-compose) (3.2.3.post2)\nInstalling collected packages: PyYAML\n Found existing installation: PyYAML 4.2b4\n\n:stderr: docker-compose 1.22.0 has requirement requests!=2.11.0,!=2.12.2,!=2.18.0,<2.19,>=2.6.1, but you'll have requests 2.20.0 which is incompatible.\nCannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.\n"}
(full paste to include all info ...)
I am using ansible provided by Fedora repo:
ansible-playbook --version
ansible-playbook 2.7.0
config file = /home/sgw/projects/provisioning-local/ansible.cfg
configured module search path = [u'/home/sgw/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 2.7.15 (default, Oct 15 2018, 15:26:09) [GCC 8.2.1 20180801 (Red Hat 8.2.1-2)]
Docker startup fails in Debian 8 when testing with dm.directlvm_device
, see test_config_storage.yml.
Working workaround is available here: moby/moby#21304 (comment)
Hello again, @haxorof,
Is ext4
the recommended fstype for CentOS for Docker? (I see that you changed over the tests from xfs
to ext4
in a recent commit.)
Ansible docker_container
module requires that either docker
or docker-py
python library is installed in order to work.
I think this would be a good thing to install these from this role as this is always required for use in dependent roles. Maybe through a variable to enable that ?
systemctl daemon-reload
must be run when toggling the configuration docker_enable_mount_flag_fix
but this is not done so this can be seen if looking at the systemctl status docker
:
May 21 22:14:51 localhost.localdomain dockerd[28725]: time="2018-05-21T22:14:51.461789807Z" level=info msg="API listen on /var/run/docker.sock"
May 21 22:14:51 localhost.localdomain systemd[1]: Started Docker Application Container Engine.
Warning: docker.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Hint: Some lines were ellipsized, use -l to show in full.
Add variable docker_daemon_config_file which shall be copied to /etc/docker/.
Use of Jinja tests used as filters has been marked deprecated in Ansible 2.5. Not possible to move to this syntax without breaking Ansible version compatibility for this role.
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|version_compare` instead use `result is version_compare`. This feature will be removed in version 2.9.
Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
If device mapper shall be used and the docker-storage-setup fails the first time it will skip several tasks next time the role is executed. This leads to Docker being setup without device mapper and it can clearly be seen when executing docker info
.
Read more about it here: https://docs.docker.com/edge/
Is replaced by nightly builds:
https://docs.docker.com/install/#nightly-builds
Task "Ensure auditd is installed" is always executed even if docker_enable_audit
is set to false
It is not possible to only use docker_daemon_config
to configure Docker daemon to for example also expose the TCP socket, see example below:
---
- hosts: localhost
roles:
- role: haxorof.docker-ce
docker_daemon_config:
hosts:
- "tcp://0.0.0.0:2376"
- "fd://"
tlsverify: true
tlscacert: "/etc/docker/ssl/ca.pem"
tlscert: "/etc/docker/ssl/cert.pem"
tlskey: "/etc/docker/ssl/key.pem"
You will see this error in the journal logs:
Jun 23 19:46:42 labrat dockerd[32221]: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [fd://], from file: [tcp://0.0.0.0:2376 fd://])
Excellent ansible role, much better than my own. I did however have a few issues, the first was I had to add:
to main-Storage.yml. Unfortunately I can't submit a pull request at present.
To get a more production ready configuration setup of Docker engine devicemapper and lvm-direct shall be used on RedHat/CentOS and Fedora. Project Atomic has a Github project called container-storage-setup
which makes it more easy to setup and recreate if the time comes to just wipe everything and start over.
Adding support to choose if this shall be used in a simple way is something which would be nice to get out-of-the-box to avoid additional steps to setup a more production like and secure environment.
See:
https://github.com/projectatomic/container-storage-setup
After upgrade of Ansible Galaxy it now seems to do some linting before import which show errors:
ERROR [role] ./tasks/bug-tweaks.yml:36:1: [error] too many blank lines (2 > 0) (empty-lines)
ERROR [role] ./tasks/checks.yml:7:1: [error] too many blank lines (1 > 0) (empty-lines)
ERROR [role] ./tasks/configure-docker.yml:64:1: [error] too many blank lines (1 > 0) (empty-lines)
ERROR [role] ./tests/prepare_storage.yml:51:1: [error] too many blank lines (1 > 0) (empty-lines)
ERROR Lint failed
Import is still successful but should fix these issues.
I had been running with edge, but I ran into this: docker/for-linux#211
What's the cleanest way to downgrade to stable, and should the role support downgrades (e.g., remove the edge repo and package and install the latest from stable)?
Docker CE stable is released every third month. The edge version of CE is released every month and this is not possible to get currently with this role.
See link below which describes how to enable edge version:
https://docs.docker.com/engine/installation/linux/fedora/#install-using-the-repository
Suggest to add configuration variable docker_enable_ce_edge
I made a change (docker_daemon_config.live-restore: true
=> false
), and the change wasn't manifest until I restarted the daemon manually.
Changes to daemon.json
should trigger a daemon restart.
I just discovered this role today. It covers a lot of bases, but it might be missing one.
I had been having issues on my host (with another Docker role, FWIW) until I added the following:
- name: 'may_detach_mounts workaround (https://github.com/moby/moby/pull/34886 & https://github.com/moby/moby/issues/34538#issuecomment-331533940)'
become: yes
copy:
content: |
fs.may_detach_mounts=1
dest: /etc/sysctl.d/may_detach_mounts.conf
register: may_detach_mounts_config
- name: activate may_detach_mounts
command: sysctl -p /etc/sysctl.d/may_detach_mounts.conf
become: yes
when: may_detach_mounts_config.changed
I see this (https://github.com/haxorof/ansible-role-docker-ce/blob/master/tasks/main-Mountflags.yml) but IIRC, I'd encountered an issue even with that set.
Thoughts?
There's one machine I'm targeting which is running Linux Mint:
"ansible_distribution": "Linux Mint",
"ansible_distribution_file_parsed": true,
"ansible_distribution_file_path": "/etc/os-release",
"ansible_distribution_file_variety": "NA",
"ansible_distribution_major_version": "18",
"ansible_distribution_release": "serena",
"ansible_distribution_version": "18.1",
I hacked up the role locally to get it to work (it's the same as Ubuntu 16.04), but I'm wondering if there's a way to override these facts, for a given host, to fool the role into thinking it's Ubuntu 16.04. Do you happen to know?
Otherwise, the role itself could support Linux Mint, but I'm not sure how much effort this project would want to put into supporting more obscure distros.
Make it possible to add packages after Docker has been installed and configured.
Executing on CentOS 7 the role fails on the conditional check below:
TASK [haxorof.docker-ce : Check if rt-beta Repository is disabled (RedHat)] ****
skipping: [localhost]
TASK [haxorof.docker-ce : Disable rt-beta Repository (RedHat)] *****************
fatal: [localhost]: FAILED! => {"msg": "The conditional check 'cmd_rhel_rt_beta_repo_disabled.rc == 1 and docker_os_dist == \"RedHat\"' failed. The error was: error while evaluating conditional (cmd_rhel_rt_beta_repo_disabled.rc == 1 and docker_os_dist == \"RedHat\"): 'dict object' has no attribute 'rc'\n\nThe error appears to have been in '/usr/share/ansible/roles/haxorof.docker-ce/tasks/setup-repository.yml': line 96, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Disable rt-beta Repository (RedHat)\n ^ here\n"}
After applying audit rules using docker_enable_audit
everything seems fine is check with Docker Bench for Security.
[INFO] 1 - Host Configuration
[PASS] 1.1 - Ensure a separate partition for containers has been created
[NOTE] 1.2 - Ensure the container host has been Hardened
[PASS] 1.3 - Ensure Docker is up to date
[INFO] * Using 17.09.0 which is current
[INFO] * Check with your operating system vendor for support and security maintenance for Docker
[INFO] 1.4 - Ensure only trusted users are allowed to control Docker daemon
[INFO] * docker:x:993:
[PASS] 1.5 - Ensure auditing is configured for the Docker daemon
[PASS] 1.6 - Ensure auditing is configured for Docker files and directories - /var/lib/docker
[PASS] 1.7 - Ensure auditing is configured for Docker files and directories - /etc/docker
[PASS] 1.8 - Ensure auditing is configured for Docker files and directories - docker.service
[INFO] 1.9 - Ensure auditing is configured for Docker files and directories - docker.socket
[INFO] * File not found
[INFO] 1.10 - Ensure auditing is configured for Docker files and directories - /etc/default/docker
[INFO] * File not found
[PASS] 1.11 - Ensure auditing is configured for Docker files and directories - /etc/docker/daemon.json
[PASS] 1.12 - Ensure auditing is configured for Docker files and directories - /usr/bin/docker-containerd
[PASS] 1.13 - Ensure auditing is configured for Docker files and directories - /usr/bin/docker-run
However after reboot (using Vagrantfile under tests) errors is seen during auditd startup so some rules are not applied.
Oct 22 14:49:56 localhost.localdomain augenrules[651]: Error sending add rule data request (No such file or directory)
Oct 22 14:49:56 localhost.localdomain augenrules[651]: There was an error in line 12 of /etc/audit/audit.rules
Oct 22 14:49:56 localhost.localdomain systemd[1]: Started Security Auditing Service.
[INFO] 1 - Host Configuration
[PASS] 1.1 - Ensure a separate partition for containers has been created
[NOTE] 1.2 - Ensure the container host has been Hardened
[PASS] 1.3 - Ensure Docker is up to date
[INFO] * Using 17.09.0 which is current
[INFO] * Check with your operating system vendor for support and security maintenance for Docker
[INFO] 1.4 - Ensure only trusted users are allowed to control Docker daemon
[INFO] * docker:x:993:
[PASS] 1.5 - Ensure auditing is configured for the Docker daemon
[PASS] 1.6 - Ensure auditing is configured for Docker files and directories - /var/lib/docker
[PASS] 1.7 - Ensure auditing is configured for Docker files and directories - /etc/docker
[PASS] 1.8 - Ensure auditing is configured for Docker files and directories - docker.service
[INFO] 1.9 - Ensure auditing is configured for Docker files and directories - docker.socket
[INFO] * File not found
[INFO] 1.10 - Ensure auditing is configured for Docker files and directories - /etc/default/docker
[INFO] * File not found
[WARN] 1.11 - Ensure auditing is configured for Docker files and directories - /etc/docker/daemon.json
[WARN] 1.12 - Ensure auditing is configured for Docker files and directories - /usr/bin/docker-containerd
[WARN] 1.13 - Ensure auditing is configured for Docker files and directories - /usr/bin/docker-runc
One way to fix this might be to add a new lines in the rules saying -c
which let auditd continue processing rules even if errors are seen.
Need understand reason why everything works before reboot directly after applying the rules etc.
dnf makecache fast
is incorrect and should be dnf makecache
.
TASK [haxorof.docker-ce : Update repository cache] *****************************
Sunday 14 October 2018 03:36:32 -0700 (0:00:00.123) 0:00:14.127 ********
fatal: [test-host]: FAILED! => {"changed": true, "cmd": "dnf makecache fast", "delta": "0:00:00.526139", "end": "2018-10-14 03:36:33.818430", "msg": "non-zero return code", "rc": 2, "start": "2018-10-14 03:36:33.292291", "stderr": "usage: dnf makecache [--legacy] [-c [config file]] [-q] [-v] [--version]\n [--installroot [path]] [--nodocs] [--noplugins]\n [--enableplugin [plugin]] [--disableplugin [plugin]]\n [--releasever RELEASEVER] [--setopt SETOPTS]\n [--skip-broken] [-h] [--allowerasing] [-b] [-C]\n [-R [minutes]] [-d [debug level]] [--debugsolver]\n [--showduplicates] [-e ERRORLEVEL] [--obsoletes]\n [--rpmverbosity [debug level name]] [-y] [--assumeno]\n [--enablerepo [repo]] [--disablerepo [repo] | --repo\n
[repo]] [-x [package]] [--disableexcludes [repo]]\n [--repofrompath [repo,path]] [--noautoremove]\n [--nogpgcheck] [--color COLOR] [--refresh] [-4] [-6]\n [--destdir DESTDIR] [--downloadonly] [--comment COMMENT]\n [--bugfix] [--enhancement] [--newpackage] [--security]\n
[--advisory ADVISORY] [--bzs BUGZILLA] [--cves CVES]\n [--sec-severity {Critical,Important,Moderate,Low}]\n [--forcearch ARCH] [--timer]\ndnf makecache: error: argument timer: invalid choice: 'fast' (choose from 'timer')", "stderr_lines": ["usage: dnf makecache [--legacy] [-c [config file]] [-q] [-v] [--version]", " [--installroot [path]] [--nodocs] [--noplugins]", " [--enableplugin [plugin]] [--disableplugin [plugin]]", "
[--releasever RELEASEVER] [--setopt SETOPTS]", " [--skip-broken] [-h] [--allowerasing] [-b] [-C]", " [-R [minutes]] [-d [debug level]] [--debugsolver]", " [--showduplicates] [-e ERRORLEVEL] [--obsoletes]", " [--rpmverbosity [debug level name]] [-y] [--assumeno]", "
[--enablerepo [repo]] [--disablerepo [repo] | --repo", " [repo]] [-x [package]] [--disableexcludes [repo]]", " [--repofrompath [repo,path]] [--noautoremove]", " [--nogpgcheck] [--color COLOR] [--refresh] [-4] [-6]", " [--destdir DESTDIR] [--downloadonly] [--comment COMMENT]", " [--bugfix] [--enhancement] [--newpackage] [--security]", " [--advisory ADVISORY] [--bzs BUGZILLA] [--cves CVES]", "
[--sec-severity {Critical,Important,Moderate,Low}]", " [--forcearch ARCH] [--timer]", "dnf makecache: error: argument timer: invalid choice: 'fast' (choose from 'timer')"], "stdout": "", "stdout_lines": []}
Executing the role with defined Docker daemon configuration fail in clean environment:
---
- hosts: localhost
roles:
- role: haxorof.docker-ce
docker_daemon_config:
init: true
Failing task:
TASK [haxorof.docker-ce : Configure Docker daemon] *********************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "checksum": "c1a367475776e627b21c8d85b9be8e74af93995c", "failed": true, "msg": "Destination directory /etc/docker does not exist"}
I have to admit that a lot of this configuration is over my head (some of it slightly, some of it completely), but I wanted to bring up a potential development.
I'm wondering if the systemd drop in for MountFlags
is still necessary, given some recent Docker patches: moby/moby#34538 (comment)
What's your take?
Add configuration option to role so it will set some default audit rules for Docker engine installation which is compliant with CIS 1.13 (https://learn.cisecurity.org/benchmarks).
Suggestion to add the following rules in file /etc/audit/rules.d/docker.rules
:
-w /usr/bin/docker -k docker
-w /var/lib/docker -k docker
-w /etc/docker -k docker
-w /usr/lib/systemd/system/docker.service -k docker
-w /usr/lib/systemd/system/docker.socket -k docker
-w /etc/default/docker -k docker
-w /var/run/docker.sock -k docker
-w /var/run/docker/libcontainerd/docker-containerd.sock -k docker
-w /etc/docker/daemon.json -k docker
-w /usr/bin/docker-containerd -k docker
-w /usr/bin/docker-runc -k docker
Some rules which is comment out in the above list does not exists in some distributions. For example in Fedora 25 with Docker CE 17.06 these files does not exist by default:
/usr/lib/systemd/system/docker.socket
/etc/default/docker
Restart of auditd might not be possible with systemctl because it needs to be restarted with the service
command: sudo service auditd restart
If restart with systemctl is done I get the following output:
Failed to restart auditd.service: Operation refused, unit auditd.service may be requested by dependency only.
See system logs and 'systemctl status auditd.service' for details.
Extend the systemd drop-in support so it will be possible to add additional systemd configuration option. This will make it possible to modify behavior for the service which is advanced configuration.
Include has been deprecated in Ansible 2.4 and will be removed in 2.8. Include needs to be replaced with include_tasks
(http://docs.ansible.com/ansible/latest/list_of_utilities_modules.html)
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for dynamic inclusions. This feature
will be removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain more about this rationale.. This
feature will be removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
If you want to for example configure proxy settings for Docker this shall be put as environment variables to the process which is described here.
In this role it now exists an environment file referenced by a systemd drop-in which can be extended to add additional environment variables like HTTPS_PROXY
etc.
https://github.com/haxorof/ansible-role-docker-ce/blob/master/templates/docker-envs.j2
When test_defaults.yml
is used during testing null
is written to daemon.json
[vagrant@localhost ~]$ cat /etc/docker/daemon.json
null
Hi
I was trying to setup devicemapper using the
docker_setup_devicemapper: true
Unfotunately this seems to skip on the following check
when: docker_stat_storage.stat.exists == false
in main-Storage and I couldn't work out where this is set.
I found the project atomic storage setup instructions http://www.projectatomic.io/docs/docker-storage-recommendation.
Using this setup should I need to do anything further like add DEVS to option to /etc/sysconfig/docker-storage-setup as part of the ansible setup.
e.g.
DEVS="/dev/vdb"
I believe the devicemapper storage can now be setup by docker from 17.06 (https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#configure-direct-lvm-mode-for-production). This works correctly If I set the following but it still doesn't seem to use the setup storage scripts.
vars
docker_enable_audit: true
docker_setup_devicemapper: true
docker_daemon_config:
icc: false
init: true
userns-remap: default
disable-legacy-registry: true
live-restore: true
userland-proxy: false
log-driver: journald
storage-driver: devicemapper
storage-opts:
- "dm.directlvm_device=/dev/sdb"
- "dm.thinp_percent=95"
- "dm.thinp_metapercent=1"
- "dm.thinp_autoextend_threshold=80"
- "dm.thinp_autoextend_percent=20"
- "dm.directlvm_device_force=false"
Just wondering if it all works for you on a clean setup
It seems that Docker uses arguments to usermod
which is not compatible with the usermod
in CentOS 7.3.1611. Have note investigated this deeper but here is what I see in the Docker daemon log during startup:
Jul 22 14:25:17 localhost.localdomain systemd[1]: Starting Docker Application Container Engine...
Jul 22 14:25:17 localhost.localdomain useradd[26488]: new group: name=dockremap, GID=991
Jul 22 14:25:17 localhost.localdomain useradd[26488]: new user: name=dockremap, UID=995, GID=991, home=/home/dockremap, shell=/bin/false
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: Error during "dockremap" user creation: Couldn't create subordinate ID ranges: Unable to add subuid range to user: "dockremap"; output: usermod: invalid option --
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: Usage: usermod [options] LOGIN
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: Options:
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -c, --comment COMMENT new value of the GECOS field
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -d, --home HOME_DIR new home directory for the user account
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -e, --expiredate EXPIRE_DATE set account expiration date to EXPIRE_DATE
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -f, --inactive INACTIVE set password inactive after expiration
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: to INACTIVE
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -g, --gid GROUP force use GROUP as new primary group
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -G, --groups GROUPS new list of supplementary GROUPS
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -a, --append append the user to the supplemental GROUPS
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: mentioned by the -G option without removing
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: him/her from other groups
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -h, --help display this help message and exit
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -l, --login NEW_LOGIN new value of the login name
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -L, --lock lock the user account
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -m, --move-home move contents of the home directory to the
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: new location (use only with -d)
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -o, --non-unique allow using duplicate (non-unique) UID
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -p, --password PASSWORD use encrypted password for the new password
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -R, --root CHROOT_DIR directory to chroot into
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -s, --shell SHELL new login shell for the user account
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -u, --uid UID new UID for the user account
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -U, --unlock unlock the user account
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: -Z, --selinux-user SEUSER new SELinux user mapping for the user account
Jul 22 14:25:17 localhost.localdomain dockerd[26482]: , err: exit status 2
Currently it always upgrades Docker to the latest version which is not always what you want. Add variable to specify if it always shall ensure latest version of Docker or not.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.