This guide will walk you through setting up TLS and TLS cert client authentication for Docker and Kubernetes.
The first step in securing Docker and Kubernetes is to set up a PKI infrastructure for managing TLS certificates.
https://github.com/cloudflare/cfssl
The CFSSL tool takes various JSON configuration files to initial a CA and produce certificates. Clone this repo and review the current set of configs and adjust them for you environment.
$ git clone https://github.com/kelseyhightower/docker-kubernetes-tls-guide.git
Before we can generate any certs we need to initialize a CA.
$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca
The Docker daemon can be protected using TLS certificates, but instead of using the openssl tools we are going to leverage our PKI from above.
$ cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=server \
docker-server-csr.json | cfssljson -bare docker-server
Results:
docker-server-key.pem
docker-server.csr
docker-server.pem
$ cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=client \
docker-client-csr.json | cfssljson -bare docker-client
Results:
docker-client-key.pem
docker-client.csr
docker-client.pem
Copy the server certs to the Docker host.
$ scp ca.pem docker-server-key.pem docker-server.pem [email protected]:~/
Move the server certs into place and fix permissions.
$ ssh [email protected]
$ sudo mv ca.pem /etc/docker/ca.pem
$ sudo mv docker-server-key.pem /etc/docker/server-key.pem
$ sudo mv docker-server.pem /etc/docker/server.pem
$ sudo chmod 0444 /etc/docker/ca.pem
$ sudo chmod 0400 /etc/docker/server-key.pem
$ sudo chmod 0444 /etc/docker/server.pem
Configure the Docker daemon to use the certs.
cat > /etc/systemd/system/docker.service <<EOF
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.io
[Service]
ExecStart=/usr/bin/docker --daemon \
--bip=10.200.0.1/24 \
--host=tcp://0.0.0.0:2376 \
--host=unix:///var/run/docker.sock \
--tlsverify \
--tlscacert=/etc/docker/ca.pem \
--tlscert=/etc/docker/server.pem \
--tlskey=/etc/docker/server-key.pem \
--storage-driver=overlay
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
Start or restart the Docker daemon
$ sudo systemctl start docker
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
Copy the client certs to the Docker client config dir.
$ mkdir -pv ~/.docker
$ cp -v ca.pem ~/.docker/ca.pem
$ cp -v docker-client.pem ~/.docker/cert.pem
$ cp -v docker-client-key.pem ~/.docker/key.pem
$ chmod 0444 ~/.docker/ca.pem
$ chmod 0444 ~/.docker/cert.pem
$ chmod 0400 ~/.docker/key.pem
$ export DOCKER_HOST="tcp://docker.kubestack.io:2376" DOCKER_TLS_VERIFY=1
$ docker ps
$ cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=server \
kube-apiserver-server-csr.json | cfssljson -bare kube-apiserver-server
Results:
kube-apiserver-server-key.pem
kube-apiserver-server.csr
kube-apiserver-server.pem
$ cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=client \
kubelet-client-csr.json | cfssljson -bare kubelet-client
Results:
kubelet-client-key.pem
kubelet-client.csr
kubelet-client.pem
$ cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=client \
kube-proxy-client-csr.json | cfssljson -bare kube-proxy-client
Results
kube-proxy-client-key.pem
kube-proxy-client.csr
kube-proxy-client.pem
$ cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=client \
kubernetes-admin-user.csr.json | cfssljson -bare kubernetes-admin-user
Results:
kubernetes-admin-user-key.pem
kubernetes-admin-user.csr
kubernetes-admin-user.pem
A Kubernetes controller includes the API server, Controller Manager, and the Scheduler.
Copy the server certs to the Kubernetes API server.
$ scp ca.pem kube-apiserver-server-key.pem kube-apiserver-server.pem [email protected]:~/
$ ssh [email protected]
$ sudo mkdir -p /etc/kubernetes/kube-apiserver
$ sudo mv ca.pem /etc/kubernetes/kube-apiserver/ca.pem
$ sudo mv kube-apiserver-server-key.pem /etc/kubernetes/kube-apiserver/server-key.pem
$ sudo mv kube-apiserver-server.pem /etc/kubernetes/kube-apiserver/server.pem
$ sudo chmod 0444 /etc/kubernetes/kube-apiserver/ca.pem
$ sudo chmod 0400 /etc/kubernetes/kube-apiserver/server-key.pem
$ sudo chmod 0444 /etc/kubernetes/kube-apiserver/server.pem
cat > policy.jsonl <<EOF
{"user":"admin"}
{"user":"scheduler", "readonly": true, "resource": "pods"}
{"user":"scheduler", "resource": "bindings"}
{"user":"kubelet", "readonly": true, "resource": "pods"}
{"user":"kubelet", "readonly": true, "resource": "services"}
{"user":"kubelet", "readonly": true, "resource": "endpoints"}
{"user":"kubelet", "resource": "events"}
EOF
$ scp policy.jsonl [email protected]:~/
$ ssh [email protected]
$ sudo mv policy.jsonl /etc/kubernetes/kube-apiserver/policy.jsonl
$ sudo chmod 0644 /etc/kubernetes/kube-apiserver/policy.jsonl
Start the Kubernetes Controller containers
$ docker-compose -p kubernetes -f compose-controller.yaml up -d
A Kubernetes worker includes the kubelet and the proxy.
Copy the client certs to the worker node.
$ scp ca.pem [email protected]:~/
$ scp kube-proxy-client-key.pem kube-proxy-client.pem [email protected]:~/
$ scp kubelet-client-key.pem kubelet-client.pem [email protected]:~/
$ ssh [email protected]
$ sudo mkdir -p /etc/kubernetes
$ sudo mv ca.pem /etc/kubernetes/ca.pem
$ sudo mv kube-proxy-client-key.pem /etc/kubernetes/kube-proxy/client-key.pem
$ sudo mv kube-proxy-client.pem /etc/kubernetes/kube-proxy/client.pem
$ sudo mv kubelet-client-key.pem /etc/kubernetes/kubelet/client-key.pem
$ sudo mv kubelet-client.pem /etc/kubernetes/kubelet/client.pem
$ sudo chmod 0444 /etc/kubernetes/ca.pem
$ sudo chmod 0400 /etc/kubernetes/kube-proxy/client-key.pem
$ sudo chmod 0444 /etc/kubernetes/kube-proxy/client.pem
$ sudo chmod 0400 /etc/kubernetes/kubelet/client-key.pem
$ sudo chmod 0444 /etc/kubernetes/kubelet/client.pem
Copy the kubeconfigs for the kubelet and proxy services.
$ scp kubeconfigs/kubelet.kubeconfig [email protected]:~/
$ scp kubeconfigs/proxy.kubeconfig [email protected]:~/
$ ssh [email protected]
$ sudo mv kubelet.kubeconfig /etc/kubernetes/kubelet/
$ sudo mv proxy.kubeconfig /etc/kubernetes/kube-proxy/
$ sudo chmod 0444 /etc/kubernetes/kubelet/kubelet.kubeconfig
$ sudo chmod 0444 /etc/kubernetes/kube-proxy/proxy.kubeconfig
$ kubectl config set-cluster secure \
--certificate-authority=ca.pem \
--embed-certs=true \
--server=https://kube-apiserver.kubestack.io:6443
$ kubectl config set-credentials admin \
--client-key=kubernetes-admin-user-key.pem \
--client-certificate=kubernetes-admin-user.pem \
--embed-certs=true
$ kubectl config set-context secure \
--cluster=secure \
--user=admin
$ kubectl config use-context secure
$ kubectl cluster-info
$ kubectl get cs
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent