Comments (9)
https://birthday.play-with-docker.com/cli-formating
Docker has organized commands into a docker
docker --help List out the nouns
docker noun --help List out the verbs or actions that can be performed by noun.
There is a format flag, we can type any raw string to simply output it, and then instructions to be parsed are included within {{ }}.
Variables injected into our template are prefixed with a dot '.'.
--format '{{json .}}'
//Provides all the options available/
--format '{{json .}}' | jq .
Beautify the output to more readable form by passing the output to jq
Simple Formatting
--format '{{.VARIABLE_NAME}}'
Access the variable in from the output
Table Formatting
If we want to display more columns, and with the table layout, we can use the table syntax with \t between each column:
docker container ls -a --format 'table {{.ID}}\t{{.Image}}\t{{.Command}}\t{{.Status}}'
We can add more complicated logic with the template. Lets take a look at the environment variable list for the last run container and print each of the elements of the PATH variable on separate lines.
docker container inspect --format '{{range .Config.Env}}{{with split . "="}}{{if eq (index . 0) "PATH"}}{{range ( split (index . 1) ":" ) }}{{printf "%s\n" .}}{{end}}{{end}}{{end}}{{end}}' $(docker container ls -lq)
from til.
Went through this resource along with the tutorial
docker run busybox
// Exists with empty output
docker run -it busybox sh
//Run busy box and access sh shell
docker run busybox echo "hello from busybox"
With all the above commands, Docker client finds the image (busybox in this case), loads up the container and then runs a command in a new container each time.
docker rm $(docker ps -a -q -f status=exited)
//Delete all the containers exited
docker container prune
//Does the same
from til.
Docker Context: allows you to connect to remote docker instances.
export TGT_HOST=10.0.0.4
// Set environment variable
ssh-keyscan -H $TGT_HOST >~/.ssh/known_hosts
// Trust the host
Connecting to Nodes with DOCKER_HOST // PREVIOUS WAY TO DO
DOCKER_HOST=ssh://$TGT_HOST docker container ls
The DOCKER_HOST variable has various syntaxes, including tcp:// to connect to an export port, and unix:/// to connect to a local socket. The easiest method to configure securely uses ssh as we did above. Connecting via ssh requires:
- Access to login with ssh to the remote host.
- The ability to run docker commands on that host as your user (typically configured by adding your user to the docker group on the remote host).
- Docker client release 18.09 or newer on both the local and remote host
Using Docker Context //The new way to do with 19.03 docker release
unset DOCKER_HOST
// Make sure to unset this environment variable
docker context create term2 --description "Terminal 2" --docker "host=ssh://$TGT_HOST"
// Create new docker context
docker context use term2
// Switch and use new context named term2
docker --context default container ls
// Run docker command in other contexts.
from til.
docker exec -it <container name> /bin/bash
// Get a bash shell in the container
docker exec -it <container name> <command>
// Execute command in the container
Do not automatically restart the container when it exits. This restart policy is the default. Therefore both commands are the same.
docker run app
docker run --restart=no app
Only restart the container if it exits with a non-zero code.
docker run --restart=on-failure app
Same as above, but limit the number of restart attempts to 10.
docker run --restart=on-failure:10 app
Always restart the container, regardless of the exit code. On Docker daemon startup, only start the container if it was already running before.
docker run --restart=unless-stopped app
Always restart the container, regardless of the exit code. The container will also start when the Docker daemon starts, even if it was in a stopped state before.
docker run --restart=always app
Resources:
- https://maximorlov.com/pm2-to-docker-automatic-restarts
- https://maximorlov.com/from-pm2-to-docker-cluster-mode
from til.
https://birthday.play-with-docker.com/jenkins-docker-hub
Ran docker container for .net core app.
Created a token in dokcer hub.
Registered the token in Jenkins.
git clone https://github.com/sixeyed/pi.git && cd pi && git checkout bday7
docker-compose -f jenkins.yml up -d
docker container run -d -p 80:80 pi:bday7 -m web
from til.
https://birthday.play-with-docker.com/run-as-user
Summary:
Create a user inside the container image.
Tell Docker to use this user.
Create and configure permissions on any user writable directories.
Configure the application to write to user writable directories.
Take extra precautions for any host volumes.
Configure any listening ports to be above 1024 inside the container.
Use docker exec args to run commands as root, rather than sudo.
QnA for preview of the content
Q. What line in a Dockerfile changes the user Docker uses to run commands?
A. USER app:app
Q. What port restrictions exists when running commands as a non-root user?
A. Commands on the host cannot be published to ports below 1024.
Q. Where do named volumes get their file permissions?
A. They are initialized with the file owner and permission from the image.
A. Existing volumes maintain the state from the previous usage.
Main objective Converting Images to Run Without Root
Docker Hub has lots of popular images that are configured to run as root. This makes them very convenient for developers looking to get started quickly with the fewest complications. However for security, it’s recommended to run our containers as a non-root user. This exercise will walk you through some of the steps needed to run containers without root.
docker image pull nginx:1.16
docker image inspect nginx:1.16
....
"User": "" // The default, when there is no user, is to **run as the root user**.
}
Create new user in the Docker file.
FROM **nginx:1.16**
RUN useradd -u 5000 app // `5000` is the user id, `app` is the user name
USER app:app // Tell docker to use the new user
Fixing file permissions
Build the new image, inspect and run container:
docker build -t user/nginx:1.16-2 .
docker image inspect user/nginx:1.16-2
docker container run --rm user/nginx:1.16-2
Now configuration issues and permission issues will appear. By default nginx
is the default user.
We need to update the nginx configuration of the container.
Extract the content from the container.
mkdir -p conf
docker container run user/nginx:1.16-2 tar -cC /etc/nginx . | tar -xC conf
This creates a conf directory that contains the nginx.conf and conf.d/default.conf.
Now update the config file.
- Edit the nginx.conf to remove or comment out the user nginx.
- Set paths for the following variables to be locations that the new user can write:
pid
client_body_temp_path
fastcgi_temp_path
proxy_temp_path
scgi_temp_path
uwsgi_temp_path
We will use /var/run/nginx/nginx.pid for the pid file, and /var/tmp/nginx/* for the other paths.
Updated nginx.conf file:
# user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
# Add the below pid
pid /var/run/nginx/nginx.pid;
events {
worker_connections 1024;
}
http {
# add the below paths
client_body_temp_path /var/tmp/nginx/client_body;
fastcgi_temp_path /var/tmp/nginx/fastcgi_temp;
proxy_temp_path /var/tmp/nginx/proxy_temp;
scgi_temp_path /var/tmp/nginx/scgi_temp;
uwsgi_temp_path /var/tmp/nginx/uwsgi_temp;
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
Updated Docker file:
FROM nginx:1.16
RUN useradd -u 5000 app \
&& mkdir -p /var/run/nginx /var/tmp/nginx \
&& chown -R app:app /usr/share/nginx /var/run/nginx /var/tmp/nginx
COPY conf/nginx.conf /etc/nginx/nginx.conf
USER app:app
Handling ports
Lets build the image with the updated configuration:
docker build -t user/nginx:1.16-3 .
And now try to run that image:
docker container run --rm user/nginx:1.16-3
This will give another permission error: nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
When applications aren’t root, they cannot listen on ports below 1024, so our web server listening on port 80 and 443 will not work. But, inside the container we can listen on a higher numbered port. And even better, with Docker we can still publish to a lower numbered port on the host, and map that high numbered port inside the container. To edit the port nginx listens on, edit the conf/conf.d/default.conf file and replace: listen 80;
with: listen 8080;
Updated Docker file:
FROM nginx:1.16
RUN useradd -u 5000 app \
&& mkdir -p /var/run/nginx /var/tmp/nginx \
&& chown -R app:app /usr/share/nginx /var/run/nginx /var/tmp/nginx
COPY conf/nginx.conf /etc/nginx/nginx.conf
USER app:app
COPY conf/conf.d/default.conf /etc/nginx/conf.d/
Rebuild image:
docker build -t user/nginx:1.16-4 .
Run that image detached, with the port mapping, and a container name:
docker container run -d -p 80:8080 --name nginx user/nginx:1.16-4
Using Volumes
Create a folder and new index.html file, and mount the folder
docker container stop nginx
docker container rm nginx
docker container run -d -p 80:8080 --name nginx -v "$(pwd)/html:/usr/share/nginx/html" user/nginx:1.16-4
We get a permission denied again. We can see that the files on the host and inside the container are owned by the same UID/GID and with the same permissions. There’s no mapping from the host user to the container user when running on Linux. There are a few ways to handle this:
Ensure the UID/GID of files on the host matches those of the container user.
Fix permissions inside the image, and only use named volumes which initialize from the image contents.
Avoid using host volumes on directories where the container will write.
Note that named volumes are only initialized when they are first created, so you only want to use these for persistent data, and not the contents of the image that you want to update with each new image.
Sudo Access
One last challenge users face when switching away from running everything as root is getting sudo access inside the container. Try to run an apt command and see what happens:
docker container exec -it nginx apt-get update
That fails without root access. If we try running sudo inside the container, what happens:
docker container exec -it nginx sudo apt-get update
Images are minimal, shipping only with the needed commands. And in containers sudo is not needed since it would be a security vulnerability (what’s the point of running as a user if that user can sudo to root) and there are better ways to get root inside of a container. The docker container exec command runs our command in the container namespace with the same settings like environment variables, working directory, and user, that the docker container run command uses to start the container. However, the docker container exec command gives options to override those settings, have a look at the help output to see how we can change the user:
docker container exec --help
Try running an apt-get update command inside the container as root instead of our app user.
Solution
docker container exec -it --user root nginx apt-get update
Extra Difference in run, exec and attach
docker run
starting a new docker container.
docker exec
running new things on a running container.
docker attach
connecting a running container's main process( PID 1) standard input/output/error with requesting medium standard I/O.
When a container is started using /bin/bash then it becomes the containers PID 1 and docker attach is used to get inside PID 1 of a container. So docker attach < container-id > will take you inside the bash terminal as it's PID 1 as we mentioned while starting the container. Exiting out from the container will stop the container.
Whereas in docker exec command you can specify which shell you want to enter into. It will not take you to PID 1 of the container. It will create a new process for bash. docker exec -it < container-id > bash. Exiting out from the container will not stop the container.
from til.
Remove Docker from Ubuntu
dpkg -l | grep -i docker : Give me a list of packages that contains the word “docker” in them
Stop and disable service
sudo systemctl stop/disable docker
sudo rm -rf /etc/systemd/system/docker.service.d
Remove all Docker packages.
sudo apt-get purge -y docker-engine docker docker.io docker-ce docker-ce-cli
sudo apt-get autoremove -y --purge docker-engine docker docker.io docker-ce
The “-y” flag here is to answer “yes” to the command prompt when it asks you whether to remove a package.
Remove all docker related files
sudo rm -rf /var/lib/docker /etc/docker
sudo rm /etc/apparmor.d/docker
sudo rm -rf /var/run/docker.sock
Remove group
sudo groupdel docker
Deactivate network interface and ethernet bridge, and delete
ifconfig docker0 down
brctl delbr docker0
Resource
- https://trello.com/c/c1RRgE2b/34-docker
- https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes
from til.
Log
from til.
Image Optimization
- Reduce the image size by using the flag --no-install-recommends in Dockerfiles - Article
from til.
Related Issues (20)
- Different Types Of API
- Behavioral Change Techniques
- A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems
- No More Snake Oil: Architecting Agility through Antifragility
- Jeff's Six Simple Steps to Story Mapping Book
- Leadership skills improve behavioral engagment
- Mastering Strategic Management HOT 1
- Software delivery performance indicators
- Software Design Lectures from University of Toronto
- Review on Quality Management using 7 QC Tools
- Refactoring and Cleaning Code - Presentation by Kent Beck
- Decsision Making Frameworks - DACI, RACI
- Developer Effectiveness: Optimizing Feedback Loops
- System Design Blueprint: The Ultimate Guide
- Team Structure And Software Design
- Making Aggregate in Domain Driven Design
- What is problem statement
- Microservices Communication - Sync vs Async vs Hybrid
- GraphQL Integrity Principles - Ensuring that the graph is well-defined, stable, and consistent
- Orchestration vs Choreography, a Guide to Composing Your Monolith
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from til.