brunoteixeira1996/gbackup

Script in Golang to backup and export info to Prometheus

gbackup is a go tool to help me backup useful information from various systems I am running. This could be something private but maybe this will help someone or give an idea on how to backup stuff using go.

The current backup plan is shown below but I backup the following:

• gokrazy perm partion (resides in gokrazy)
• gokrazy data folder (resides in a LXC in proxmox)
• postgresql databases (resides in a LXC in proxmox)
  • currently I am only doing the backup for the waiw and leak databases
• syncthing folder (resides in a LXC in proxmox)
• monitoring files (resides in a LXC in proxmox)
  • this is files from alertmanager, prometheus, grafana and pushgateway
• work laptop

At first I was using rsync to backup everything to an external hard drive plugged in my proxmox and then use cp to backup everything to a different location (storagepool - HDD that resides inside the proxmox). However now I am using rsync in both backups.

I was using crontab inside proxmox but now I've managed to create a similiar cron using golang so its easier to apply this to any unix based OS without caring if crontab is installed or no.

Then I monitor the rsync commands using Prometheus as shown bellow.

gbackup's Issues

Size diff

Add size diff in order to view if target is bigger or smaller

Move from cronjob

Its ok to use cronjob from Linux, however I rather use everything in Go instead.
Knowing this, I should implement something like this from gmah

        runCh := make(chan struct{})
        go func() {
                // Run forever, trigger a run at 18:00 every day.
                for {
                        now := time.Now()
                        runToday := now.Hour() < 18
                        today := now.Day()
                        log.Printf("now = %v, runToday = %v", now, runToday)
                        for {
                                if time.Now().Day() != today {
                                        // Day changed, re-evaluate whether to run today.
                                        break
                                }


                                nextHour := time.Now().Truncate(time.Hour).Add(1 * time.Hour)
                                log.Printf("today = %d, runToday = %v, next hour: %v", today, runToday, nextHour)
                                time.Sleep(time.Until(nextHour))


                                if time.Now().Hour() >= 18 && runToday {
                                        runToday = false
                                        runCh <- struct{}{}
                                }
                        }
                }
        }()


        for range runCh {
                run(args.Email, args.Password, args.Dump, &newMessages, args.Gokrazy)
        }

Add gmah backup

Add gmah backup into gokrazy perm

Is cp good?

I should use rsync to copy from /mnt/pve/external to /storagepool/backups since that will be faster and rsync will only make diffs

Time duration

Add time duration

Incremental backups?

Right now I am performing incremental backups with rsync but I want to have similar information not incremental.
Need to change some flags on rsync

Run on demand

Create a route that when a POST is made perform the backup on demand

Email without info

After using cronjob from golang email template is empty without Content

Implement new way of rsync for gk LXC

I dont need to have a cronjob inside gk LXC. I can have a similar setup like Syncthing.
I need to create a ssh key file in proxmox, then add that to the authozired_keys inside gk lxc and then make sure I can rsync from gk lxc to proxmox.
This way I will have information in grafana.
I should do the same for the leaks.

• Implement correct rsync mechanism to gk LXC c59c851
• Implement correct rsync mechanism to leaks 51a36eb
• Implement correct rsync mechanism to monitoring 23e1ffb
• Implement correct rsync mechanism to postgresql fe95fd0

backup to storagepool

For some reason the backup for storagepool is not working

Fix TODO

I've noticed there are some typos and stuff I dont use anymore

Clean README

Update README with the new setup

clean function not executing

Looks like there are tars older than 15 days still available in storagepool and external hard drive

https://github.com/BrunoTeixeira1996/gbackup/blob/984ec18cc9aa77bb7e8b50d89f4f3bd2893791b1/targets/leaks.go#L50C1-L51C1

Execute backups on demand

Create simple webserver that waits for a POST request to execute a backup on demand.
This webserver has a GET request to list the available backup targets meaning that the POST requests expects a JSON object of targets or in case of all a *
Later this can be used in the telegram bot