veerendra2/raspberrypi-homeserver

A collection of applications and tools to make awesome Raspberry Pi homerserver

Raspberry Pi Homeserver

A collection of self-host docker swarm stacks

PiHole	Grafana	Prometheus	Portainer	Homer
ufw	Filebrowser	Jellyfin	Nextcloud	Traefik
Wireguard	qBittorrent	Radarr	Sonarr	Jackett
SearXNG	Vaultwarden	rest-server (Restic)	IT-Tools

Docker Swarm

Table of Contents

• Features
• My Raspberry Pi Config
• Architecture Diagram
• Getting Started
• Related Blogs
• GitBook Docs
• Project Roadmap
• Youtube Video

Features

• Automated with Ansible
• All services are deployable on docker swarm
• Traefik reverse proxy
• Grafana dashboards to view
  • System metrics
  • Internet speed everyone hour
  • Uptime with blackbox exporter
  • Docker container metrics
• Homer dashboard to view all services
• Uncomplicated firewall
• qBittorrent with Wireguard proxy and kill switch with healthcheck

My Raspberry Pi Config

Model	Raspberry Pi 4 Model B Rev 1.4
CPU	BCM2835 (4) @ 1.800GHz
Memory	8 GB
OS	Ubuntu 22.04.2 LTS aarch64
Case	Geekworm NASPi Gemini 2.5 V2.0 Dual 2.5 Inch SATA HDD/SSD
Disks	- 1 TB Crucial SSD - 1 TB Western Digital HDD
Docker Swarm Nodes	1 (Single node docker swarm cluster)

Dashboard

Architecture Diagram

Getting Started

Refer Gitbook Docs for more details and how to deploy manually

Ansible automation works well, if all services are intended to deploy on single box. Since I have only one machine, don't have to worry about the docker volumes, all of my docker volumes stored on single SSD mounted to host.

• Follow prerequisite manual steps to prepare Pi
• Browse vars.yml to configuration

$ git clone https://github.com/veerendra2/raspberrypi-homeserver.git
$ cd raspberrypi-homeserver
# review vars.yml and inventory.yml and run
$ ansible-playbook main.yml

NOTE

• This setup is created to deploy all services on single node docker swarm cluster. If you want to use this setup on multi node swarm cluster, there are some additional tweaks required like
  • Change placement

    ...
      deploy:
        replicas: 1
        placement:
          constraints: [node.role == manager]
    ...

• Currently I'm using duckdns sub-domain (Refer example -> services/traefik/docker-stack.yml#32). Refer my blog post to get more info.
• Update docker volume mount paths.

Related Blogs

• Portainer vs Yacht
• Pi-hole with DHCP Relay in Docker
• Wireguard VPN and BitTorrent on Docker Swarm (Part 1)
• Wireguard VPN and BitTorrent on Docker Swarm (Part 2)
• Traefik HTTPS Config with DuckDNS for Local Homeserver
• How to Deploy rest-server(Restic) on Docker Swarm Behind Traefik Reverse Proxy

raspberrypi-homeserver's Issues

Anible config and inventory

• Create default ansible config file
• Create inventory file(For localhost too)

Create fan controller script

• https://wiki.geekworm.com/X-C1_Software
• Fan script: https://github.com/geekworm-com/x-c1/blob/main/fan.py

Add `vaultwarden`

• https://github.com/dani-garcia/vaultwarden/tree/main/docker
• https://github.com/dani-garcia/vaultwarden/wiki/Proxy-examples
• https://github.com/dani-garcia/vaultwarden/wiki/Using-the-PostgreSQL-Backend

HTTPS with DuckDNS

Resources

• https://www.youtube.com/watch?v=qlcVx-k-02E
• https://go-acme.github.io/lego/dns/duckdns/
• https://doc.traefik.io/traefik/https/acme/#providers
• https://www.reddit.com/r/docker/comments/p82vgr/traefik_with_lets_encrypt_using_duckdns_domain/

Nextcloud is slow

• Noticed nextcloud is slow even redis is emabled.
• From https://pant.github.io/nextcloudpi-test-docs/en_Why-is-my-Pi-so-slow.html, understood, Ethernet and USB shared same BUS

Review and deploy `immich`

• https://github.com/immich-app/immich

Jellyfin Deployment

• Jellyfin deployment via docker-compose

Links

• Jellyfin Installation
• https://hub.docker.com/r/linuxserver/jellyfin

Wiki Pages

Write documentation in wiki pages for all apps

Disaster Recovery Test [Nextcloud]

• Test and create backup and restore steps for nextcloud

Test nextcloud setup with Android app

Add `adminer`

Review and add adminer for postgresql DB

• https://hub.docker.com/_/adminer/
• https://www.adminer.org/

Smoke tests tasks after install

• Ansible tasks to run smoke tests to see service is up

Make Anisible Role for each app

• Review and create ansible role for each app
• docker-compose.yml should generate from jinja template with variables like
  • docker image version
  • UID & GID
  • hostname and container_name

Move docker volume to disks

• Move all docker volumes to disk
• It should have disaster recoverable

Ignore "Host not found" in nginx location

Ngnix is failing to start if any of specified location is down

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2022/12/27 20:36:03 [emerg] 1#1: host not found in upstream "pihole" in /etc/nginx/conf.d/locations/pihole.conf:2
nginx: [emerg] host not found in upstream "pihole" in /etc/nginx/conf.d/locations/pihole.conf:2
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh

Solutions

• proxy_read_timeout (https://stackoverflow.com/questions/18740635)
  • Need to set upstream
• try_files?

Add `searxng`

• https://github.com/searxng/searxng

Add `rest-server`

• https://github.com/restic/rest-server
• https://github.com/restic/rest-server/blob/master/examples/compose-with-grafana/docker-compose.yml

Overclock Pi in Ubuntu on-demand

Miscellaneous bugs

• * Generate ssh key pair
• * become: yes for disabling bluetooth
• * Fix traefik name while deploying
• * Check apps are installed and then install
• * Windows VS code ssh explorer

Yacht vs Portainer

• https://github.com/portainer/portainer
  • Agent - Server Architecture
  • Comparable with K8s, Swarm
  • Security features
  • CE is open source
• https://github.com/SelfhostedPro/Yacht
  • Young project
  • Simple
  • No security and user management

Test Pihole

• Test Pihole in LAN
• Configure devices to use PiHole in automated way or check for manual

NOTE

• Our router doesn't support add DNS server
• Does it required to DHCP server?

Apps in v2.0

• Dynamic DNS
  • Duck DNS
  • Cloudflare DDNS(https://github.com/timothymiller/cloudflare-ddns, https://hub.docker.com/r/oznu/cloudflare-ddns)
• Bittorret with VPN
• Nextcloud
• OpenWRT

Migrate to Docker Swarm Services

• https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/

Add Jackett

Integrate Bunkerweb and DuckDNS

• Integrate Bunkerweb(With Authelia)
• DuckDNS sub domain

Review file system and RAID setup

[GitOps] CI/CD with Github self-hosted runners

• Deploy Github self-hosted runner agent on RPi4(And Ansible)
• Setup CI/CD

[QA] raspberrypi-homeserver v3.0 release

• Deployments
• Application functionality

Web file browser Deployment

• Web file browser Deployment via docker-compose

Links

https://github.com/filebrowser/filebrowser

docker and ufw

docker iptables is not Honoring ufw rules
Workarounds

• moby/moby#4737 (comment) (Tested!)
• https://stackoverflow.com/questions/30383845/what-is-the-best-practice-of-docker-ufw-under-ubuntu
• https://p1ngouin.com/posts/how-to-manage-iptables-rules-with-ufw-and-docker

#4

Set `UID` and `GID` for all apps (Configured userns-remap )

Set UID and GID instead of running as root

Can't connect to Jellyfin server in Android

Add `calibre-web`

• https://github.com/janeczku/calibre-web
• https://hub.docker.com/r/linuxserver/calibre-web

Create common databases script in postgresql

• init script to create multiple databases and users

Review `vars` and `secrets`

• Review vars and secrets.
• Check which vars and secrets are moved and placed
• Check directory structure, try to keep simple

Bug fixes

• Bug fixes
• Add disk mount

Switch to Trafik Proxy

• https://geek-cookbook.funkypenguin.co.nz/docker-swarm/traefik/
• https://github.com/traefik/traefik
• https://doc.traefik.io/traefik/

Update Images

pihole-FTL: no process found

veerendra@atom:~/projects/raspberrypi-homeserver/apps/pihole$ sudo docker-compose up
...
  [✓] Creating new gravity databases
pihole    |   [i] Storing downloaded domains in new gravity database...Stopping pihole-FTL
pihole    |
pihole    |   [✗] Unable to fill gravity table in database /etc/pihole/gravity.db_temp
pihole    |
  [✓] Cleaning up stray matter
pihole    |   [✗] DNS service is NOT running
pihole    | Stopping pihole-FTL
pihole    | pihole-FTL: no process found
pihole    | Stopping pihole-FTL
pihole    | pihole-FTL: no process found
...

Fix docker ufw issue

https://stackoverflow.com/questions/30383845/what-is-the-best-practice-of-docker-ufw-under-ubuntu

Add `watchtower`

• https://containrrr.dev/watchtower/

Refactor repo

• Rename repo
• Make services generic(Remove hardcoded values and make genetric)
• Change README.md and update documentation
• Change ansible automation. Make genetric

Stablize apps deployments for v1

Below are apps picked for v1

• Docker container management tool
• PiHole
• Prometheus monitoring setup(With Internet monitoring)
• Homer

nginx reverse proxy(https://hub.docker.com/r/linuxserver/swag)

Improve `fan.py` script

The existing fan.py seems not efficient. While setting fan speed, the fan goes off and start again.

Some fan scripts

• https://github.com/folkhack/raspberry-pi-pwm-fan-2
• https://github.com/tedsluis/raspberry-pi-pwm-fan-control

Split nginx config for each `location`

Add `emulatorjs`

https://emulatorjs.org/

Add `Bettercap UI`

• https://www.bettercap.org/usage/webui/

Nextcloud

• Nextcloud deployment with docker-compose
• Test drive access with external USB HDD

Power optimization tricks for RaspberryPi

• https://blues.io/blog/tips-tricks-optimizing-raspberry-pi-power/
• https://di-marco.net/blog/it/2020-04-18-tips-disabling_bluetooth_on_raspberry_pi/

Wireguard VPN in docker

Wireguard and Deluge torrent clinet