lisenet/kubernetes-homelab

Building a multi-master multi-node Kubernetes homelab with kubeadm, Ansible, Helm and Terraform.

kubernetes-homelab's Issues

Update Packer to use Rocky 8.7

Use Terraform variables to define software versions where possible

Disable Kubecost product analytics

Add CKA study notes

Update to Kubernetes 1.26 and Istio 1.17

Istio 1.17 has been tested with Kubernetes release 1.26.

https://istio.io/latest/docs/releases/supported-releases/#supported-releases/

Reorganise folder structure

Introduce logic for grouping directories together and making the structure easier to use.

Update to Kubernetes 1.24 and Istio 1.14

Istio 1.14 has been tested with these Kubernetes releases: 1.21, 1.22, 1.23, 1.24.

Use Trivy Operator

Install Trivy Operator to continuously scans Kubernetes cluster for security issues, and generates security reports as Kubernetes Custom Resources.

https://aquasecurity.github.io/trivy-operator/latest/

Rebuild OpenVPN container image for v2.5.8

Migrate to Rocky Linux 9

It would be good to understand if we can run Kubernetes on Rocky Linux 9.

Update broken links after updating folder structure

Migrate OpenVPN Docker config to its own repository for better maintenance

Consider adding Velero for cluster backups

Update Packer to use Rocky 8.6

Rocky 8.6 has been released:

https://rockylinux.org/news/rocky-linux-8-6-ga-release/

Add CKAD study notes

security query

Out of curiosity your repo seems littered with keys and passwords. Is that intentional ? I'm assuming they're different in your environment, but was just curious.

Cheers

Add Kubernetes cluster configuration section to documentation

What is used for CRI, CNI, CSI, backups etc?

Remove 3DES ciphers from Kubernetes apiserver

Kubernetes apiserver accepts vulnerable ciphers:

TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (SWEET32)
TLS_RSA_WITH_3DES_EDE_CBC_SHA (SWEET32)

Nmap scane results.

Nmap scan report for kubelb.hl.test (10.11.1.30)
Host is up (0.0017s latency).

PORT     STATE SERVICE
6443/tcp open  sun-sr-https
| ssl-cert: Subject: commonName=kube-apiserver
| Subject Alternative Name: DNS:kubelb.hl.test, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.apps.hl.test, DNS:srv31, IP Address:10.96.0.1, IP Address:10.11.1.31
| Issuer: commonName=kubernetes
| Public Key type: rsa
| Public Key bits: 2048
| Signature Algorithm: sha256WithRSAEncryption
| Not valid before: 2023-07-05T15:28:54
| Not valid after:  2024-07-04T15:28:54
| MD5:   b0c2 7a2e 2620 be38 645f ee8a 79e6 ed29
|_SHA-1: 46ef 7bcd c3af 3786 1a8e 3887 ab78 7b89 e2fb cd32
| ssl-enum-ciphers: 
|   TLSv1.2: 
|     ciphers: 
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp256r1) - C
|       TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
|     compressors: 
|       NULL
|     cipher preference: server
|     warnings: 
|       64-bit block cipher 3DES vulnerable to SWEET32 attack
|_  least strength: C

Nmap done: 1 IP address (1 host up) scanned in 0.79 seconds

Update CKA, CKAD and CKS study notes to use Kubernetes v1.28

Integrate Terrascan with Pre-commit

https://runterrascan.io/docs/integrations/pre-commit-integration/

Add stale action

Inactive issues/PRs should be automatically closed to reduce man hours.

Packer SSH failed

2022/12/03 18:53:53 packer-builder-qemu plugin: [INFO] Attempting SSH connection to 127.0.0.1:3687...
2022/12/03 18:53:53 packer-builder-qemu plugin: [DEBUG] reconnecting to TCP connection for SSH
2022/12/03 18:53:53 packer-builder-qemu plugin: [DEBUG] handshaking with SSH
2022/12/03 18:53:53 packer-builder-qemu plugin: [DEBUG] SSH handshake err: ssh: handshake failed: read tcp 127.0.0.1:60552->127.0.0.1:3687: read: connection reset by peer
Cancelling build after receiving interrupt
==> rocky8: Deleting output directory...
2022/12/03 18:53:57 Cancelling builder after context cancellation context canceled
2022/12/03 18:53:57 packer-post-processor-shell-local plugin: Received interrupt signal (count: 1). Ignoring.
2022/12/03 18:53:57 packer-builder-qemu plugin: Received interrupt signal (count: 1). Ignoring.
2022/12/03 18:53:57 packer-builder-qemu plugin: [WARN] Interrupt detected, quitting waiting for SSH.
2022/12/03 18:53:57 packer-builder-qemu plugin: failed to unlock port lockfile: close tcp 127.0.0.1:5979: use of closed network connection
2022/12/03 18:53:57 packer-builder-qemu plugin: failed to unlock port lockfile: close tcp 127.0.0.1:3687: use of closed network connection
2022/12/03 18:53:57 packer-builder-qemu plugin: [DEBUG] SSH wait cancelled. Exiting loop.
2022/12/03 18:53:57 [INFO] (telemetry) ending rocky8
==> Wait completed after 6 minutes 4 seconds

I'm trying to follow your packer build process but I'm also lazy and don't want uncompressed iso's all over the shop. I copied most of your packer config and just substituted

    "iso_urls": "https://dl.rockylinux.org/vault/rocky/8.6/isos/x86_64/Rocky-8.6-x86_64-boot.iso",

All seemed to work, except it fails SSH. Could you provide any advice please ?

Update CKA study notes for Kubernetes v1.26

speedtest.ConfigRetrievalError: HTTP Error 403: Forbidden

Dockerised speedtest fails with the following error:

Running a speedtest using default server
Traceback (most recent call last):
  File "/usr/local/bin/speedtest-to-influxdb.py", line 23, in <module>
    s = speedtest.Speedtest()
  File "/usr/local/lib/python3.10/site-packages/speedtest.py", line 1095, in __init__
    self.get_config()
  File "/usr/local/lib/python3.10/site-packages/speedtest.py", line 1127, in get_config
    raise ConfigRetrievalError(e)
speedtest.ConfigRetrievalError: HTTP Error 403: Forbidden

Hashicorp Vault to store K8s secrets

Enable TLS encryption for InfluxDB

Prometheus error opening storage failed: lock DB directory: resource temporarily unavailable

When a new version of Prometheus is rolled out, Kubernetes performs a RollingUpdate, meaning that a new pod is started before the old one terminates. This seems to cause issues with Prometheus.

Prometheus has a database lock on the NFS server that should be released before the new pod can write to the DB.

It would make sense to use a different deployment strategy if uptime isn't critical.

Update to Kubernetes 1.28 and Istio 1.19

Kubernetes 1.28 comes with updated version of etcd v3.5.9 which supports TLS 1.3. Support has been added in v3.5.8.

Update to Kubernetes 1.25 and Istio 1.16

Istio 1.16 has been tested with Kubernetes release 1.25.

https://istio.io/latest/docs/releases/supported-releases/#supported-releases/

Add more RAM to KVM servers

Hypervisors could use more RAM, including Kubernetes worker nodes.

Migrate to Kubernetes Community-Owned Package Repositories

https://kubernetes.io/blog/2023/08/15/pkgs-k8s-io-introduction/#how-to-migrate

Replace the yum repository definition so that yum points to the new repository instead of the Google-hosted repository. Make sure to replace the Kubernetes minor version in the command below with the minor version that you're currently using:

    cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
    [kubernetes]
    name=Kubernetes
    baseurl=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/
    enabled=1
    gpgcheck=1
    gpgkey=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/repodata/repomd.xml.key
    exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
    EOF

Add SonarQube deployment

https://www.sonarsource.com/products/sonarqube/

Say Farewell to CentOS 7

With less than a year left (June 30, 2024) until CentOS 7 becomes EOL, it's time we say farewell to it, as no new deployments will use the OS.

Terraform should not parse any variables that are defined in Kubernetes YAML files

Terraform parsing of loaded templates should be disable because not all templates are for Terraform to evaluate.

Use Loki for log aggregation

Consider adding Packer

Provisioning VMs using PXE is nice and works well, but takes time. It would be good to add Packer as an option for libvirt guests.

Rebuild OpenVPN image from main repository

Replace repository http://dl-cdn.alpinelinux.org/alpine/edge/testing/ with http://dl-cdn.alpinelinux.org/alpine/edge/main/.

Update to Kubernetes 1.29, Calico 3.27 and Istio 1.20

https://github.com/projectcalico/calico/releases/tag/v3.27.0

https://istio.io/latest/news/releases/1.20.x/announcing-1.20.2/

Rebuild OpenVPN container image for v2.6.0

https://github.com/OpenVPN/openvpn/releases/tag/v2.6.0

Consider adding Argo CD for application deployment

Update Calico to v3.23

VPN access to the Kubernetes cluster from the Internet

Add CKS study notes