kelseyhightower/certificate-init-container

Bootstrap TLS certificates for Pods using the Kubernetes certificates API.

Certificate Init Container

The certificate-init-container generates TLS certificates for pods using the Kubernetes certificate API.

See the current release for usage details.

Prerequisites

• Kubernetes 1.6.0+

Usage

Create a deployment that uses the certificate-init-container:

kubectl create -f deployments/tls-app.yaml

The certificate-init-container will generate a private key, certificate signing request (csr), and submit a certificate signing request to the Kubernetes certificate API, then wait for the certificate to be approved.

kubectl get pods

NAME                       READY     STATUS     RESTARTS   AGE
tls-app-2342064067-c9xwf   0/1       Init:0/1   0          5s

View the certificate-init-container logs for more details:

kubectl logs tls-app-2342064067-c9xwf -c certificate-init-container

2017/04/06 06:58:02 wrote /etc/tls/tls.key
2017/04/06 06:58:02 wrote /etc/tls/tls.csr
2017/04/06 06:58:02 waiting for certificate...
2017/04/06 06:58:02 certificate signing request (tls-app-2342064067-c9xwf-default) not approved; trying again in 5 seconds
2017/04/06 06:58:27 certificate signing request (tls-app-2342064067-c9xwf-default) not approved; trying again in 5 seconds

List the certificate signing requests and locate the csr pending for the tls-app pod:

kubectl get csr

NAME                               AGE       REQUESTOR                               CONDITION
tls-app-2342064067-c9xwf-default   1m        system:serviceaccount:default:default   Pending

Review the csr details:

kubectl describe csr tls-app-2342064067-c9xwf-default

Name:                   tls-app-2342064067-c9xwf-default
Labels:                 <none>
Annotations:            <none>
CreationTimestamp:      Thu, 06 Apr 2017 06:17:16 -0700
Requesting User:        system:serviceaccount:default:default
Status:                 Pending
Subject:
        Common Name:    10-228-0-10.default.pod.cluster.local
        Serial Number:
Subject Alternative Names:
        DNS Names:      10-228-0-10.default.pod.cluster.local
                        example.com
                        tls-app.default.svc.cluster.local
        IP Addresses:   10.228.0.10
                        127.0.0.1
Events:	<none>

Approve the pending certificate signing request:

kubectl certificate approve tls-app-2342064067-c9xwf-default

certificatesigningrequest "tls-app-2342064067-c9xwf-default" approved

Once the certificate signing request has been approved the certificate-init-container will fetch the signed certificate and write it to a shared filesystem.

kubectl logs tls-app-2342064067-c9xwf -c certificate-init-container

2017/04/06 06:58:02 wrote /etc/tls/tls.key
2017/04/06 06:58:02 wrote /etc/tls/tls.csr
2017/04/06 06:58:02 waiting for certificate...
2017/04/06 06:58:02 certificate signing request (tls-app-2342064067-c9xwf-default) not approved; trying again in 5 seconds
2017/04/06 06:58:27 certificate signing request (tls-app-2342064067-c9xwf-default) not approved; trying again in 5 seconds
...
2017/04/06 07:00:28 wrote /etc/tls/tls.crt

Next the certificate-init-container will exit and the pod will start the remaining containers which will have access to the certificate and private key.

kubectl get pods

NAME                       READY     STATUS    RESTARTS   AGE
tls-app-2342064067-c9xwf   1/1       Running   0          2m

Create a service for the tls-app deployment to view the certificate details.

kubectl expose deployment tls-app --type=LoadBalancer

Current Release

Container Image:

gcr.io/hightowerlabs/certificate-init-container:0.0.1

See the example deployment for more details.

Usage:

certificate-init-container -h

Usage of certificate-init-container:
  -additional-dnsnames string
    	additional dns names; comma separated
  -cert-dir string
    	The directory where the TLS certs should be written (default "/etc/tls")
  -cluster-domain string
    	Kubernetes cluster domain (default "cluster.local")
  -hostname string
    	hostname as defined by pod.spec.hostname
  -namespace string
    	namespace as defined by pod.metadata.namespace (default "default")
  -pod-ip string
    	IP address as defined by pod.status.podIP
  -pod-name string
    	name as defined by pod.metadata.name
  -service-ips string
    	service IP addresses that resolve to this Pod; comma separated
  -service-names string
    	service names that resolve to this Pod; comma separated
  -subdomain string
    	subdomain as defined by pod.spec.subdomain

certificate-init-container's Issues

ship an automated approver for general usecases

Capturing thoughts from an email thread:

I think it would possible to leverage
the confidentiality of the service account token to do this e.g. an
approval controller will approve these dual client/server certs if and
only if:

1. the pod that is the subject of the CSR is running as the service
   account that created the CSR.
2. the IPs in the SAN are a subset of the pod IP and service IPs of
   the services the pod backs.
3. the hostnames in the SAN are a subset of the pod hostname and
   service DNS names of the services the pod backs.
4. the pod isn't requesting to be in any RBAC groups.

Add flags to set organizational unit and serial number

I have a use-case that requires me to set the OU (Organizational Unit) and SN (Serial Number). It'd be nice to have a flag that lets me set the OU, and a flag that lets me set the SN or have the SN autogenerated.

Add flag to overwrite common name

I have a use-case that requires me to specify a bespoke common name (CN) that is different from the first DNS name (dnsName[0]). It'd be nice to have a flag that let's me overwrite the common name so that it's different from the first DNS name.

Allow headless FQDN to be CN

Right now the CN will always be the 'default' ip-based CN, but some client certificate auth schemes use only CN to authenticate (namely: http://floragunncom.github.io/search-guard-docs/tls_certificates_production.html). To facilitate glob matching like: *.subdomain.namespace.svc.cluster.local on CN, I would suggest having a flag that switches the array places of the headless service hostname and the pod ip hostname. Will submit PR soon.

resource not namespaced

I am getting an error while trying to load this init container in my deployment:

2018/06/14 12:32:05 wrote /etc/tls/tls.key
2018/06/14 12:32:05 wrote /etc/tls/tls.csr
2018/06/14 12:32:05 unable to create the certificate signing request: resource not namespaced 

Here is my deployment yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: users
  labels:
    app: users
spec:
  replicas: 1
  selector:
    matchLabels:
      app: users
  template:
    metadata:
      labels:
        app: users
    spec:
      initContainers:
        - name: certificate-init-container
          image: gcr.io/hightowerlabs/certificate-init-container:0.0.2
          imagePullPolicy: Always
          env:
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
          args:
            - "-additional-dnsnames=comply-online.com"
            - "-cert-dir=/etc/tls"
            - "-namespace=$(NAMESPACE)"
            - "-pod-ip=$(POD_IP)"
            - "-pod-name=$(POD_NAME)"
            - "-service-names=users"
          volumeMounts:
            - name: tls
              mountPath: /etc/tls
      containers:
      - name: users
        imagePullPolicy: Always
        image: compliance/compliance-users
        args:
          - "-tls-cert=/etc/tls/tls.crt"
          - "-tls-key=/etc/tls/tls.key"
        readinessProbe: # Check when application is ready
          httpGet:
            path: "/api/v1/ready"
            port: nodejs-port
          initialDelaySeconds: 15
          timeoutSeconds: 5
        livenessProbe: # healthcheck every 60 seconds
          httpGet:

              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
          args:
            - "-additional-dnsnames=comply-online.com"
            - "-cert-dir=/etc/tls"
            - "-namespace=$(NAMESPACE)"
            - "-pod-ip=$(POD_IP)"
            - "-pod-name=$(POD_NAME)"
            - "-service-names=users"
          volumeMounts:
            - name: tls
              mountPath: /etc/tls
      containers:
      - name: users
        imagePullPolicy: Always
        image: compliance/compliance-users
        args:
          - "-tls-cert=/etc/tls/tls.crt"
          - "-tls-key=/etc/tls/tls.key"
        readinessProbe: # Check when application is ready
          httpGet:
            path: "/api/v1/ready"
            port: nodejs-port
          initialDelaySeconds: 15
          timeoutSeconds: 5
        livenessProbe: # healthcheck every 60 seconds
          httpGet:
            path: "/api/v1/ready"
            port: nodejs-port
          initialDelaySeconds: 15
          timeoutSeconds: 10
          periodSeconds: 60
        ports:
        - name: nodejs-port
          containerPort: 3000
        - name: sslport
          containerPort: 3443
        volumeMounts:
          - name: tls
            mountPath: /etc/tls
      - name: mongo
        image: mongo
        ports:
        - name: mongo
          containerPort: 27017
          hostPort: 27017
        volumeMounts:
          - name: mongo-persistent-storage
            mountPath: /data/db
      volumes:
        - name: tls
          emptyDir: {}
        - name: mongo-persistent-storage
          emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: users
  labels:
    app: users
spec:
  selector:
    app: users
  ports:
  - port: 3000
    protocol: TCP
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: mongo
spec:
  ports:
    - port: 27017
      targetPort: 27017
  selector:
    app: mongo
  type: ClusterIP

Which namespace is it referring to?

Pod is stuck in Init:CrashLoopBackOff

kubectl create -f tls-app.yaml

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
tls-app-dc7787455-69rhx 0/1 Init:Error 1 6s
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
tls-app-dc7787455-69rhx 0/1 Init:CrashLoopBackOff 1 12s
$ kubectl logs tls-app-dc7787455-69rhx
Error from server (BadRequest): container "tls-app" in pod "tls-app-dc7787455-69rhx" is waiting to start: PodInitializing

I'm just running the yaml uploaded in keley's github repository. Am I missing something?

getting empty tls.crt

context: kubernetes 1.6.0, mongodb stateful set with certificate-init-container

problem: after approving the csr request and a successful exit of the init container, mongo pod has mounted the tls directory, in which a valid key file appears to be present, but the corresponding crt file appears to be empty.

here is the result of executing commands in that container:

Leifs-MacBook-Pro:devops leif$ kubectl --namespace=www exec mongo-0 --container mongo ls /etc/tls
tls.crt
tls.csr
tls.key
Leifs-MacBook-Pro:devops leif$ kubectl --namespace=www exec mongo-0 --container mongo cat /etc/tls/tls.crt
Leifs-MacBook-Pro:devops leif$ kubectl --namespace=www exec mongo-0 --container mongo cat /etc/tls/tls.key
-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQDgOIfy6cg0QOa5xdsHqVaTQUMNjx3noKrzIhFcZjkhyEDHBGg7
q79vL412QGh/J4jABDDBlBtJYht6Joif9/CWsJDK8nukCtIUGAkanfqN9LYw6Kx0
+zekhD4VYuqBkdHhX4gPergQEx2KPwYTiMcopHU7ZjcWHhg5KwjKDxWyYQIDAQAB
AoGBAMMw80iIM/0pMrIuh8pXpEJ04cZ//GyT9b3KvopNgxbhmWKnd4XascA9CU0U
LOPkqqtd+oPKRybO/VSNDwsTcKBcH+By2apnruV5Fyq7ajZ9p9FcfKWB8ULJpTGV
ZvxWJmlSN1sBxzHRavxfYUhPxKMta2cAoDcqgd2x6JXg8/txAkEA4dtD9/pS59e0
vfsBl7pN6aobMXFTcs5fNi43u/PgA1BKR8B1uEb79+DqZdlTx74eGQ8rjfA6cJzF
E1fmBXqqdwJBAP4lYUY9DZKBH/m9hw4pe51s1PS/4ysDNklzBFsFRO/mjZhlWEc3
EhfpqIHCC958D2Y7zyPlwejHkoQt/jewZ+cCQEDFrEFyxdH9l7UgRj2oTW3kvfW5
BXCYZM/aA1vgve7VWOgo9tnJabuWAkMBR7ix+lm/cwM60Ne47f/1Jl+N5VMCQCvy
wNiJlEBCbgoc2S78w3CcslLEn/7/JjMbXaRzy9EEeHXjCJoADvPtpwMRUEYgA/W0
4AQgJiEf6dlLrstoiP8CQFMhrjexarF7GkqwdL5uHD+3V8EdkVEh9LPKcEX2zO9e
r6ULfXqfGJrDmH25MRCD3P6AYVubWzQbCUmpCxM1esA=
-----END RSA PRIVATE KEY-----

happy to provide more context or the associated yaml files