aflyingcougar/gitops

GitOps principles to define kubernetes cluster state via code. Community around k8s@home is on discord: https://discord.gg/RGvKzVg

It's My k8s in a Box

---

📖  Overview

This repo is my Kubernetes cluster in a declarative state. Flux and Helm Operator watch my clusters folder and makes the changes to my cluster based on the yaml manifests. Renovate auto updates images and helm charts based on upstream changes.

Feel free to join our Discord if you have any questions.

---

⚓  k8s Distro

Currently using k3s by way of a customized template-cluster-k3s ansible playbook.

---

🚤  Deploying k3s

1. pip install pipenv
2. pipenv install
3. pipenv run gilt overlay
4. pipenv run ansible-playbook -i provision/ansible/inventory/inventory.yaml provision/ansible/playbooks/k3s-install.yaml
5. k label nodes k-node1.crutonjohn.com k-node2.crutonjohn.com k-node3.crutonjohn.com k-node4.crutonjohn.com k-node5.crutonjohn.com crutonjohn.com/rook=true
6. kubectl taint nodes k-master01.crutonjohn.com k-master02.crutonjohn.com k-master03.crutonjohn.com crutonjohn=control-plane:NoSchedule

Installing and bootstrapping flux

1. Have a working kubeconfig

2. Have flux installed

3. Have GITHUB_TOKEN env var set to a Github PAT

4. To boostrap the cluster:

    flux bootstrap github \
    --components=source-controller,kustomize-controller,helm-controller,notification-controller \
    --path=clusters/env/production \
    --version=latest \
    --owner=crutonjohn \
    --repository=gitops
   

5. sops -d sops-secret.enc.yaml | kubectl apply -f -

if you need storage :)

kustomize build clusters/core/pvc/

---

💻  Hardware Configuration

All my nodes below are running bare metal Ubuntu 20.04.x

Device	Count	OS Disk Size	Data Disk Size	Ram	Purpose
Raspberry Pi 4	3	120GB (USB Booting SSD)	N/A	4 GB	k8s Control Plane
HP 800 G3 Mini	3	1x 120GB SSD	1x 1TB M.2 (rook-ceph)	32GB	k8s Workers
Dell 7040 Micro	2	1x 500B HDD	1x 1TB M.2 SSD (rook-ceph)	32GB	k8s Workers

💻  Supporting Infrastructure

Device	Count	OS Disk Size	Data Disk Size	Ram	Purpose
Supermicro CSE-512B	1	2x 500GB Spinning Rust	N/A	32GB	ZFS on Linux Host
Xyratex HB-2425	1	N/A	24x 1TB SSD	N/A	ZFS Disk Shelf

---

📝  IP addresses

This table is a reference to IP addresses in my deployments and may not be fully up-to-date

Deployment	Address
traefik-ingress (external)	192.168.130.100
traefik-ingress (internal)	192.168.130.101
syncthing-discovery	192.168.130.104
syslog-ng	192.168.130.106
home-assistant	192.168.130.108
emqx	192.168.130.109
scrypted	192.168.130.110

---

🤝  Community

Thanks to all the people who donate their time to the Kubernetes @Home community. Join us at https://discord.gg/k8s-at-home

A lot of inspiration for my cluster came from the people that have shared their clusters over at awesome-home-kubernetes