criticalstack/crit

A tool for bootstrapping Kubernetes

crit's Issues

Embed additional service/package files

Files, such as the kubelet drop-in file 20-crit.conf, can be embedded and templated so that they can be included in the binary and used on systems that do not install from the deb/rpm package (which includes it). This drop-in is a requirement to ensure that the kubelet dynamic configuration (created here) that crit writes out will be loaded by the kubelet systemd service.

For the kubelet drop-in file, crit could potentially check if the file is present when running crit up and copy it in when:

1. It detects that the kubelet service is installed
2. The drop-in does not already exist

I imagine this will be complicated slightly by differences in linux distros with varying standard locations for systemd service files, so this will have to be taken into account.

It is also possible that crit should be able to install other files related to the service, such as the service unit file itself. Any ideas on files that can be included that will help provide a better user experience are extremely welcome.

Files can be added to the templates/ directory and then will be built-in with the crit binary.

CVE-2020-26160 (High) detected in github.com/dgrijalva/jwt-go-dc14462fd58732591c7fa58cc8496d6824316a82

CVE-2020-26160 - High Severity Vulnerability

Vulnerable Library - github.com/dgrijalva/jwt-go-dc14462fd58732591c7fa58cc8496d6824316a82

Golang implementation of JSON Web Tokens (JWT)

Dependency Hierarchy:

• github.com/labstack/echo/v4/middleware (Root Library)
  • ❌ github.com/dgrijalva/jwt-go-dc14462fd58732591c7fa58cc8496d6824316a82 (Vulnerable Library)

Found in HEAD commit: 8d66617fe1b0b461f9d8db18671163e72fc71420

Found in base branch: main

Vulnerability Details

jwt-go before 4.0.0-preview1 allows attackers to bypass intended access restrictions in situations with []string{} for m["aud"] (which is allowed by the specification). Because the type assertion fails, "" is the value of aud. This is a security problem if the JWT token is presented to a service that lacks its own audience check.

Publish Date: 2020-09-30

URL: CVE-2020-26160

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

---

Step up your Open Source Security Game with WhiteSource here

"crit create token" returns no output

When creating a bootstrap token using crit create token, no output is returned to the user:

$ crit create token
$

This is inconvenient for the following reasons:

• There is no indication that the command was successful.
• The user is left to search for the token using kubectl get secrets. This becomes difficult when there are several tokens:

$  kg secrets
NAME                                             TYPE                                  DATA   AGE
...
bootstrap-signer-token-dzqwn                     kubernetes.io/service-account-token   3      161m
bootstrap-token-8459vn                           bootstrap.kubernetes.io/token         6      4m10s
bootstrap-token-drwuzf                           bootstrap.kubernetes.io/token         6      2m14s
bootstrap-token-epw5fz                           bootstrap.kubernetes.io/token         6      31m
bootstrap-token-otgi8h                           bootstrap.kubernetes.io/token         6      2m3s
bootstrap-token-pxsarz                           bootstrap.kubernetes.io/token         6      3m2s
bootstrap-token-vygvaf                           bootstrap.kubernetes.io/token         6      2m11s
...

• The user has to base64 decode and concatenate <token-id>.<token-secret>:

$ kg secret bootstrap-token-8459vn -o yaml 
apiVersion: v1
data:
  auth-extra-groups: c3lzdGVtOmJvb3RzdHJhcHBlcnM6Y3JpdDpkZWZhdWx0LW5vZGUtdG9rZW4=
  expiration: MjAzMC0xMC0wNVQxOTozODo1M1o=
  token-id: ODQ1OXZu
  token-secret: eXE0azkzZHhkd3Zxaml2cQ==
  usage-bootstrap-authentication: dHJ1ZQ==
  usage-bootstrap-signing: dHJ1ZQ==
kind: Secret
...

"crit certs renew" fails with default certificate directory structure

Given the following directory structure (default behavior of crit up), the crit certs renew command fails:

/etc/kubernetes/
├── admin.conf
├── controller-manager.conf
├── kubelet.conf
├── scheduler.conf
├── pki
│   ├── apiserver-healthcheck-client.crt
│   ├── apiserver-healthcheck-client.key
│   ├── apiserver-kubelet-client.crt
│   ├── apiserver-kubelet-client.key
│   ├── apiserver.crt
│   ├── apiserver.key
│   ├── auth-proxy-ca.crt
│   ├── auth-proxy-ca.key
│   ├── ca.crt
│   ├── ca.key
│   ├── front-proxy-ca.crt
│   ├── front-proxy-ca.key
│   ├── front-proxy-client.crt
│   ├── front-proxy-client.key

$ crit certs renew
2020-10-07 17:58:33.228354 I | open /etc/kubernetes/pki/admin.conf: no such file or directory

$ crit certs renew  --cert-dir /etc/kubernetes
2020-10-07 18:17:48.063614 I | open /etc/kubernetes/ca.crt: no such file or directory

nfpms deprecation of files in config

https://goreleaser.com/deprecations#nfpmsfiles

This will need to be updated before 2021-06-21 to ensure that it still works.

Cinder throws errors if $HOME/.kube does not exist

cinder create cluster and cinder export kubeconfig will both fail if there is not already a $HOME/.kube directory created.

Output here:

❯ cinder create cluster
Creating cluster "cinder" ...
 🔥  Generating certificates
 🔥  Creating control-plane node
 🔥  Installing CNI
 🔥  Installing StorageClass
 🔥  Running post-up commands
<output elided>
2020/10/06 10:49:46 open /home/brooks/.kube/config.lock: no such file or directory

❯ cinder export kubeconfig
2020/10/06 10:54:56 open /home/brooks/.kube/config.lock: no such file or directory

Should be a simple fix to check for existence and create if not found to avoid the error

Add `crit down` command/functionality

Just as the crit up command is used to bootstrap a new node, the crit down sub-command should be added that stops and cleans up that node. This mostly involves using the cri-api to list and stop all containers running on the node, and stopping the kubelet service. The protobuf file specifying the runtime service can be seen here and usage of the cri-api within crit can be demonstrated here:

• https://github.com/criticalstack/crit/blob/master/pkg/kubernetes/remote/remote.go

• crit/pkg/cluster/init.go

  Lines 73 to 98 in d9fce24

  	r, err := remote.NewRuntimeServiceClient(ctx, cfg.NodeConfiguration.ContainerRuntime.CRISocket())
  	if err != nil {
  	return err
  	}
  	// The apiserver container is only ever queried once, because the
  	// presumption is made that it should not have to restart during
  	// initial bootstrapping. Should this no longer be the case in the
  	// future, this will need to be adapted to account for scenarios where
  	// the intended flow of the apiserver allows for 1 or more container
  	// restarts.
  	var container *runtimeapi.Container
  	if err := wait.PollImmediateUntil(500*time.Millisecond, func() (bool, error) {
  	var err error
  	container, err = r.GetContainerByName(ctx, "kube-apiserver")
  	if err != nil {
  	return false, nil
  	}
  	return true, nil
  	}, ctx.Done()); err != nil {
  	return err
  	}
  	status, err := r.GetContainerStatus(ctx, container.GetId())
  	if err != nil {
  	return err
  	}

I do not believe any files should be removed as part of crit down (not that I can think of currently), but a good litmus test for functionality should be that a user can run crit up again after running crit down and it will bootstrap a new node just as it did initially.

Disable anonymous-auth by default

At this time crit follows precedent from kubeadm for enabling anonymous requests. Since crit uses a healthcheck-proxy sidecar to safely expose health checks via a limited role, the API server could potentially just default to anonymous-auth=false.

The one reason why this may not be desired is that it must put the health check on a different port than the API server. So I think some considerations will need to be made as to whether this is a good idea or not.

Something to explore would also be using the sidecar proxy as the API server port and forward all traffic to the internal API server port, while only reverse proxying with the limited role for the /health endpoint.