kovtalex/k8s-pi

Building a Raspberry Pi Kubernetes Cluster on Ubuntu using kubeadm and Containerd

Changing current ubuntu password

ssh ubuntu@<ip>

Adding new user

ssh ubuntu@<ip>
sudo adduser k8s-user

Adding Privilege Escalation

sudo usermod -aG sudo k8s-user

logout

Creating a key pair on client server

ssh-keygen -t rsa

Copying public key to the remote server

ssh-copy-id k8s-user@<ip>

Updating users

ssh k8s-user@<ip>
sudo usermod -s /bin/nologin ubuntu

Updating sshd config

sudo vi /etc/ssh/sshd_config

PermitRootLogin no
PasswordAuthentication no

sudo systemctl reload sshd

Updating OS

sudo apt update && sudo apt dist-upgrade -y

Configuring network

sudo vi /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg

network: {config: disabled}

sudo vi /etc/netplan/01-netcfg.yaml

network:
  ethernets:
    eth0:
      dhcp4: no
      addresses:
        - <ip>/24
      gateway4: 192.168.0.1
      nameservers:
        addresses: [192.168.0.1, 8.8.8.8]
        search: [domain.local]
  version: 2

sudo vi /etc/netplan/50-cloud-init.yaml

Delete all text apart from the comments

sudo netplan apply

sudo hostnamectl set-hostname pimaster

sudo hostnamectl set-hostname piworker - for worker node

sudo vi /etc/hosts

<control plane ip> pimaster
<worker ip> piworker

Checking iptables version

iptables --version

Should be iptables v1.8.4 (legacy)

OR

sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
sudo update-alternatives --set arptables /usr/sbin/arptables-legacy
sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy

Updating boot command

cgroup="$(head -n1 /boot/firmware/cmdline.txt) cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1"
echo $cgroup | sudo tee /boot/firmware/cmdline.txt

sudo reboot

cat /proc/cmdline

Disabling swap

ssh k8s-user@<ip>

sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

containerd prerequisites

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

Installing containerd

sudo apt install containerd -y

sudo mkdir -p /etc/containerd

sudo containerd config default | sudo tee /etc/containerd/config.toml
sudo systemctl restart containerd

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"

sudo apt install kubeadm kubelet kubectl -y

Bootstrapping master

sudo kubeadm config images pull
sudo kubeadm init

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Installing CNI plugin

curl https://docs.projectcalico.org/manifests/calico-typha.yaml -o calico.yaml
kubectl apply -f calico.yaml

Bootstrapping worker

sudo kubeadm join <control plane ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

Checking

kubectl get nodes -o wide

Usefulness

Join the additional node after the init token expired

kubeadm token create --print-join-command

Installing containerd CLI

wget https://github.com/containerd/containerd/releases/download/v1.5.4/containerd-1.5.4-linux-amd64.tar.gz
tar xvf containerd-1.5.4-linux-amd64.tar.gz
cd bin

sudo ctr --namespace k8s.io container list