1copenut/homelab-learning

Building a homelab is hard work. I'm taking notes :)

homelab-learning's Issues

Add Netdata to Proxmox

It's a fantastic monitoring tool, and looks easy to install! https://www.youtube.com/watch?v=fYl5poBJtE4

Develop an automated hardening routine for Ubuntu images

• https://vectops.com/2020/01/provision-proxmox-vms-with-ansible-quick-and-easy/
• https://www.the-digital-life.com/protect-your-linux-server/

Explore adding Squid proxy to pfSense

https://turbofuture.com/internet/Intercepting-HTTPS-Traffic-Using-the-Squid-Proxy-in-pfSense

Pi My Life Up

• #28
• #3

Start using nmap for basic port scanning

Port scanning. It's a neutral activity, all in the intent. These seem like good starting points:

• https://medium.com/dsckiit/ethical-hacking-101-getting-started-with-nmap-cb51757a0a85
• https://www.youtube.com/watch?v=4t4kBkMsDbQ

Start working Portswigger's online academy for ethical hacking

Portswigger Academy has free courses for learning websec. I think it'd be good to start with the section on XSS cross-site scripting.

Start building a list for moving the homelab into a 4U rack space

So now that the rack is up and running, it might be worth moving the homelab tower into a 2U or 3U 4U rack and stacking it.

Has to be a 4U to fit an ATX style power supply unit.

Add all the pfSense plugins to make things better

This video by Lawrence Systems has a great rundown of the basics to add:
https://www.youtube.com/watch?v=oOWjHeqbWUE

Configure pfSense as the new firewall

Description

Install the pfSense firewall on new APU hardware. Ensure you have installed it with ZFS--pfSense does not like power loss and ZFS is a better way to keep things running smoothly.

Guide

• https://hometechhacker.com/6-pfsense-configurations-to-do-after-install/
• https://teklager.se/en/knowledge-base/installing-pfsense-apu-boards-over-serial/

Tasks

• Configure DHCP
• Lock down access to pfSense
• Add pfBlocker
• Stand up OpenVPN
• Traffic shaping
• Backups
• Email alerts

Diagram the LAN and OPT1 ports, switches, and VLANs

Failing to plan is planning to fail. Diagram out your flow of traffic from firewall ports to switches, and how you might divide VLANs.

Consider things like:

• LAN being for management only
• OPT1 being all VLANs
• "Lagging" LAN and OPT1 together for redundancy

Consider running my own DNS server with Unbound on the Pi-hole device

Pi-hole still relies on an upstream DNS provider like Cloudflare (1.1.1.1) or Google (8.8.8.8) and maybe that's okay. Or maybe I want to be my own DNS server. Unbound might be one way to get there. Things to consider:

• Is Unbound going to slow down my network?
• Will a Pi Zero run Pi-hole and Unbound without bogging down?
• Do I need or want this level of privacy?

Consider adding a redundant Pi-hole server

If I'm running Pi-hole for ad sinks, does it make sense to add a second, redundant Pi-hole server? Maybe it does. Considerations:

• Is it worth having a second Pi-hole server for DNS fallback?
• Does my network need this level of redundancy?
• What's the cost tradeoff of upstream providers like Cloudflare (1.1.1.1) vs. the cost of buying and maintaining another server?

Tune pfSense for highest throughput

Not sure I'll ever get true 1GB throughput for the money I'm spending, but this guide makes it a whole lot better:
https://teklager.se/en/knowledge-base/apu2-1-gigabit-throughput-pfsense/

Install new fans for the rack

The fans were way too noisy to run in an office. So left a couple panels open for now and ordered new ones.

Get the CompTIA A+ certification

After talking to J last night, there's a ton of good stuff and should not be skipped. So...

• Buy the A+ study guide
• Study
• Schedule a test
• Profit

Start consuming logs for real endpoints in Elastic

Quick start

• https://www.elastic.co/training/logging-quick-start

Integrations

• firewall
• endpoints (all OS)
• devices?

Set up new rack and switch

Description

This will serve as the meta (over-arching) project for setting up a new switch and rack for the home network.

Tasks

• #10
• #11
• #30
• #33
• #31
• #32

Write a blog post about building the new rack

You've learned a lot from this adventure. Let's write about it. If nothing else, it pushes you to get the website pushed to prod already. :)

Rework firewall rules to include a broad RFC1918 deny all

There's already good rules in place, but consolidation makes it easier to reason about.
https://help.ui.com/hc/en-us/articles/115010254227-UniFi-USG-Firewall-How-to-Disable-InterVLAN-Routing#option%202

Confirm existing items are working before provisioning the new switch

Do this like you'd work on a pull request. Make a small change, confirm it works, then move on. For this task:

• Power down the Cloud Key
• Power down the USG, switch, and modem
• Move modem into the cage
• Run power for the homelab to the outlet
• Plug modem, big switch into the surge protector
• Plug USG, small switch into the battery backup
• Bring everything back up, confirm:
• Internet works
• Controller works
• Radios work
• Then start provisioning the new switch

Stand up Nginx Reverse Proxy Manager

I should set up Nginx Reverse Proxy Manager. This will need a remote server to properly enable Let's Encrypt SSL.

• https://www.youtube.com/watch?v=P3imFC7GSr0
• https://nginxproxymanager.com/

Set up a WireGuard VPN

The time has come to run my own VPN. WireGuard sounds nice:
https://www.paolotagliaferri.com/wireguard-vpn-tunnel-with-pfsense-ce-2-5-2-package/

Stand up the Kali wireless card and ensure monitor mode

May need to try out disabling the primary wifi card, or switching to the USB adapter

Install PuTTY for Ubuntu

Will need a way to serial into the switch. This will be that way: https://www.ssh.com/academy/ssh/putty/linux

Consider relocating the server rack

So it's nice having the rack close by, but also noisy. Might consider a new space to locate it, if the room isn't too dusty. It's close t the central A/C, so it'll never get hot.

Set up Raspberry Pi syslog server

Description

Start capturing syslog traffic for the network on a Pi so we can consume it locally and remotely. Use these articles for a starting point and be mindful the SD card won't stand up to multiple writes like you're going to have with noisy log files.

• https://nwmichl.net/2020/03/15/telegraf-influxdb-grafana-as-syslog-receiver/
• https://nwmichl.net/2020/07/14/telegraf-influxdb-grafana-on-raspberrypi-from-scratch/
• https://jamesachambers.com/best-ssd-storage-adapters-for-raspberry-pi-4-400/
• Ableconn USB to M2.SATA adapter - $28

Tune Proxmox backups and snapshots

https://www.dlford.io/managing-proxmox-how-to-home-lab-part-2/ has a lot of good resources for making sure Proxmox stays up to date and software does too.

Send pfSense logs to Elastic

Once pfSense is up and running, let's get the logs outsourced to Elastic and start reviewing them.

• https://toddysm.com/2020/08/26/learn-more-about-your-home-network-with-elastic-siem-part-2-collect-firewall-logs/
• https://docs.netgate.com/pfsense/en/latest/monitoring/logs/remote.html
• https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html
• https://docs.elastic.co/en/integrations/pfsense

Consider if the logs are transmitted encrypted or clear text, whatever option you settle on.

Install Pi-hole and start using it for DNS

Start blocking ad traffic from IoT devices using Pi-hole

• https://pi-hole.net/
• https://blog.codinghorror.com/an-exercise-program-for-the-fat-web/
• https://scotthelme.co.uk/securing-dns-across-all-of-my-devices-with-pihole-dns-over-https-1-1-1-1/
• https://scotthelme.co.uk/catching-naughty-devices-on-my-home-network/

Purchase hardware to install pfSense

Minibox.com sells a pretty solid PCEngines board that will more than handle what I want to do on a home network for roughly $300 plus shipping: https://www.mini-box.com/ALIX-APU-4D4-AMD-G-Series-GX-412TC

• Board
• Case
• Power supply
• M2 SSD

Explore homelab traffic

Once the SIEM and IDS are set up, it's time to start snooping around and play a little capture the flag on a private subnet. These articles ought to get me started:

• https://infosecwriteups.com/this-is-how-i-hacked-my-neighbors-computer-b4863aa8f305
• https://infosecwriteups.com/hacking-a-home-network-54e06a9d9730

Take the Elastic SIEM fundamentals training

https://www.elastic.co/training/elastic-security-fundamentals-siem

Flash APU bios to something a bit newer

APUs are solid, but the BIOS is older. Thankfully PCEngines offers an easy upgrade path:
https://teklager.se/en/knowledge-base/apu-bios-upgrade/

Add a Portainer (Docker) server to consume Netdata in Prometheus and Grafana

Since we're putting in Netdata in #50, might as well see what we can derive from it with better visualizations: https://www.youtube.com/watch?v=uimGcQVRaqI

It'll also be a chance to try out Portainer and Docker containers in a low-risk way.

Finish hardware purchasing

Description

Need to purchase the rack, tools, and cabling for setting up a new switch.

Enable promiscuous mode traffic for SIEM VM

Proxmox doesn't allow promiscuous mode traffic on network bridges by default. The SIEM (ELK stack, Graylog) need this feature enabled to work properly. Found this article that seemed promising: https://monach.us/operations/sending-promisc-traffic-within-proxmox/

This will actually require me to mirror the OVS bridge in the regular Linux bridge when the time comes. The SIEM VM will live on the same bridge as pfSense and mirror the bridge with my offensive and vuln boxes.

Consider moving the controller to a Raspberry Pi

• https://brendonmatheson.com/2020/03/13/unifi-controller-on-raspberry-pi.html
• https://tynick.com/blog/09-08-2019/unifi-controller-with-raspberry-pi-and-docker/
• https://hub.docker.com/r/jacobalberty/unifi/

Ingest threat data with the Threat Intel Filebeat module

Description

Learn more about threat data when you're up and running with FileBeats: https://elastic.github.io/security-research/whitepapers/2021/07/01.threat-intel-filebeat-module/article/

Install security information and even management (SIEM) VM and get it consuming basic log data

• Select a SIEM stack (ELK stack or Graylog are the front runners)
• Enable Snort or Suricata on the pfSense VM
• Pull logs into the SIEM
• Visualize firewall logs

Take a look at Sophos Home Firewall

Might be an alternative to pfSense, or something to noodle around with on the lab.
https://www.youtube.com/watch?v=7pvgKc3WdEg

Explore OpenTelemetry with Elastic

I'm still not super clear what telemetry means, but watched a video about Open Telemetry last weekend and want to add it to the homelab for exploration.

Take inventory of electricity usage

This might be interesting as a way to track electricity usage as I add servers and equipment: https://grafana.com/blog/2021/04/15/learn-how-to-monitor-your-energy-use-at-home-with-a-raspberry-pi-grafana-and-prometheus/

Provision the new switch

A few things:

• Plug it in, turn it on
• Find the local IP in the controller
• Map the VLAN names and numbers. You'll need them.
• Log in to the switch and run through setup
• Log out, and confirm login through the serial port (may need to download putty or something similar)
• Stand up the needed ports. This includes:
• Six on the right, three on the left.
• Three on the left should be for radios and a hard-line for work.
• Confirm the radios work
• Have a beer. You earned it.

Add Swarm and Portainer to your remote Docker server

By running a server on a subdomain like research or labs I can try out Docker, Docker Swarm, and Portainer for orchestration in a relatively safe, low-cost way.

• Stand up Docker
• Stand up Swarm
• Stand up Portainer
• Start adding services like Nginx, Recon NG, Spiderfoot. Why not start an OSINT lab?
• Grab a referral code for getting started https://www.youtube.com/watch?v=P3imFC7GSr0

Harden Pi Zero before adding Pi-hole

Found some really good articles on hardening:

• https://pimylifeup.com/raspberry-pi-security/
• https://pimylifeup.com/raspberry-pi-ssh-keys/
• https://pimylifeup.com/setup-2fa-ssh/

Start using Wireshark for packet capture

Wireshark and pcap seem to be the way to analyze traffic, at least getting started. Let's put it into practice.

Consider adding a second Pi-Zero for redundant DNS

I could always stick with the Cloudflare DNS (1.1.1.1) for redundancy, but it'd be cool to be completely in-house.

Install Suricata and start those logs

Suricata is next. Get it up and running, then start moving logs to Elastic for ingestion.

• https://www.youtube.com/watch?v=KRlbkG9Bh6I
• https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-suricata.html
• https://hurricanelabs.com/splunk-tutorials/your-all-in-one-guide-to-setting-up-pfsense-and-suricata-in-splunk/ (not Splunk tho)

Set up rack and cabling for new switch

Description

Punch down cable endpoints, build the rack, start provisioning the switch, running cable

Transfer my domain to something more...better

It's time to move away from legacy domain registrar to a better one. Options include:

• Namecheap
• Cloudflare