Giter Site home page Giter Site logo

snap-sync's Introduction

snap-sync

This bash script sends incremental snapshots to another drive for backing up data. Plug in and mount any btrfs-formatted device you want your system to be backed up to (like a USB drive). When you run the script you will be prompted to select a mounted btrfs device, or you can optionally select the disk using its UUID on the command line.

The script iterates through all snapper configurations by default (this can be changed using the -c flag). For each configuration it creates a new local snapshot. If you have never synced to the specified device you will be prompted to enter a directory on the device where the backup snapshots will go. Additionally you are shown the location of the backed up snapshot. If you have performed a backup to this device before, only the changes since the last backup have to be sent.

Installation

Arch Linux

Install the snap-sync package using pacman if using Arch Linux.

Fedora

Install the snap-sync package from this Copr created by @brndd.

Manual

Download the latest release and signature from the releases page, verify the download, and then run make install. snapper is required.

If your system uses a non-default location for the snapper configuration file, specify it on the command line with SNAPPER_CONFIG. For example, for Arch Linux use

make SNAPPER_CONFIG=/etc/conf.d/snapper install

Starting with release 0.6, the tarballs are signed with my key with fingerprint F7B28C61944FE30DABEEB0B01070BCC98C18BD66 (public key). Previous tarballs and commits used a different key with fingerprint 8535CEF3F3C38EE69555BF67E4B5E45AA3B8C5C3.

Documentation

See snap-sync(8) after installation.

Troubleshooting

After reviewing the man page, check the issues page and file a new issue if your problem is not covered.

Contributing

Help wanted! Feel free to fork and issue a pull request to add features or tackle an open issue.

Related projects

See @rzerres's fork which has several enhancments.

snap-sync's People

Contributors

wesbarnett avatar kalbasit avatar albertmichaelj avatar tlvince avatar nicohood avatar r1do avatar thomwiggers avatar

Stargazers

Max Chernoff avatar Piotr Całus avatar  avatar eplord avatar  avatar MstrPi avatar Mark Arranz avatar hal avatar  avatar rprihoda avatar ǃшɒʞɒH ǃǀɄ avatar  avatar unknowntrojan avatar  avatar Daniel Fahey avatar Eduardo Leggiero avatar  avatar Asseel Naji avatar Mohammad Reza Mokhtarabadi avatar Jonas avatar Lorenz Sieben avatar TwoPinkyNoBrain avatar  avatar Benjamin Steenhoek avatar Gipo avatar  avatar Iizuki avatar  avatar simone viozzi avatar Rafed Ramzi avatar Dmitry Mozzherin avatar Robert Kubosz avatar Ryan C. Thompson avatar Nikolay Gniteev avatar Frederic Crozat avatar عثمان محامدي avatar Kshamendra avatar Kyle Moad avatar  avatar Ishan Choudhari avatar Paul George avatar M. F. W. Bink avatar Titus Tzeng avatar Eugen Trippel avatar Ananth avatar Fredrik Söderström avatar  avatar Ada avatar D. Debnath avatar Sebastian Schlatow avatar  avatar Bryce Allen avatar Scotty Trees avatar cocoonk1d avatar  avatar Simon Wydooghe avatar Gabriel Davila avatar  avatar keith avatar Hans Christian Schmitz avatar Max avatar  avatar Joseph Chiocchi avatar  avatar Giovanni avatar Fabien Bourgeois avatar  avatar  avatar umko avatar Blair Drummond avatar Gerad Munsch avatar Tony DeRocchis avatar  avatar  avatar Peter Wedder avatar Jordan Williams avatar Eugene Triguba avatar atlas-booker avatar Timo avatar Sergio de la Garza avatar  avatar Florian Freund avatar Fabio Brito d'Araujo e Oliveira avatar Sven-Eric Matthes avatar  avatar jyomu avatar Tobias Cudnik avatar ૮༼⚆︿⚆༽つ avatar Torstein Eide avatar Brian Clemens avatar Cornelius Riemenschneider avatar Manolis Kapernaros avatar Adrien Le Guillou avatar Daniel Grossmann-Kavanagh avatar Jan Trejbal avatar  avatar Dane Lipscombe avatar York-Simon Johannsen avatar  avatar Richard Thomas avatar

Watchers

 avatar Walter Rudametkin avatar  avatar Ralf Zerres avatar  avatar  avatar  avatar Lennard Gäher avatar  avatar

snap-sync's Issues

snap-sync ignores home setting

Dont ask me how i triggered this bug, but it ignored the home setting. If i abort the root setting it asks me to backup the home setting. Everything "unnormal" that I did was to try safe the home backup inside a root snapshot and then went one directory upwards. This is obviosly super stupid, but I wanted to test everything.

[alarm@alarmpi snap-sync]$ snapper -c home list
Type   | #  | Pre # | Date                     | User | Cleanup  | Description               | Userdata                                                            
-------+----+-------+--------------------------+------+----------+---------------------------+---------------------------------------------------------------------
single | 0  |       |                          | root |          | current                   |                                                                     
single | 1  |       | Fri Nov 11 19:00:11 2016 | root | timeline | timeline                  |                                                                     
single | 8  |       | Fri Nov 11 19:21:34 2016 | root |          | latest incremental backup | backupdir=////backup, uuid=6c1a3286-34c5-4bbd-a7cc-5762c8dd5ced     
single | 9  |       | Fri Nov 11 19:39:32 2016 | root |          | latest incremental backup | uuid=1e44de19-7755-4219-9884-bc932958ac9e                           
single | 10 |       | Fri Nov 11 19:53:04 2016 | root |          |                           |                                                                     
single | 11 |       | Fri Nov 11 19:54:20 2016 | root |          | latest incremental backup | backupdir=backup/root/13/, uuid=5a9dcbec-12b0-40c3-803a-dc1faac20d33
[alarm@alarmpi snap-sync]$ touch test
[alarm@alarmpi snap-sync]$ sudo snap-sync 
Select a mounted BTRFS device to backup to.
1) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/)
2) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/tmp)
3) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/log)
4) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/abs)
5) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/.snapshots)
6) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/home)
7) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/data)
8) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/srv)
9) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/cache/pacman/pkg)
10) 5a9dcbec-12b0-40c3-803a-dc1faac20d33 (/mnt)
0) Exit
Enter a number: 10

You selected the disk with UUID 5a9dcbec-12b0-40c3-803a-dc1faac20d33.
The disk is mounted at /mnt.

Will backup //.snapshots/19/snapshot to /mnt/backup/root/19//snapshot
Continue with backup [Y/n]? n
Aborting backup for this configuration.

Will backup /home/.snapshots/12/snapshot to /mnt/backup/home/12//snapshot
Continue with backup [Y/n]? n
Aborting backup for this configuration.

Done!
[alarm@alarmpi snap-sync]$ sudo snap-sync 
Select a mounted BTRFS device to backup to.
1) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/)
2) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/tmp)
3) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/log)
4) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/abs)
5) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/.snapshots)
6) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/home)
7) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/data)
8) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/srv)
9) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/cache/pacman/pkg)
10) 5a9dcbec-12b0-40c3-803a-dc1faac20d33 (/mnt)
0) Exit
Enter a number: 10

You selected the disk with UUID 5a9dcbec-12b0-40c3-803a-dc1faac20d33.
The disk is mounted at /mnt.

Will backup //.snapshots/19/snapshot to /mnt/backup/root/19//snapshot
Continue with backup [Y/n]? y
At subvol //.snapshots/19/snapshot

Cleanup service

You know how difficult it is to delete multiple btrfs snapshots in different sub(!)directories. A cleanup service similar to snapper itself would be nice. You could set similar settings as snapper does for local configs, but not a time cleanup, more a number cleanup. Or you could provide a manual command to at least cleanup the backups easier.

# List all external backups
snap-sync -c home list

# Clean external backups x-y
snap-sync -c home clean 172-185

# Another idea would be to keep the last x backups
snap-sync -c home keep 3

The last option could be also used as systemd service (which should run after the backup). Depending on the setting you then have backups fr the last 3 weeks. Keeping every month is not possible like this because this might be way to complex to implement for now, but that is up to you.

The (default) setting or the number of backups to keep should be selectable in the snapper config itself, so you can keep more copies of home than root etc. Or you can completely disable the cleanup for critical snapper configs.

Corruption prevention/log

I just started your tool for my root partition and the shell with the command disappeared (I possibly closed it). I dont know if the backup finished or not. And if it was not finished I am asking myself if snap-sync can detect it, to not produce more corrupted backups.

A logfile of the output would be nice. /var/log/snap-sync maybe. I did not run it as a service.

A detection if more than 2 snap-sync snapshots exist would be nice. the new one was possibly not transmitted correct. --> I checked what happens and it seems that it leaves the corrupted one, because its not tagged:

single | 2450 |       | Fri 23 Dec 2016 09:00:05 PM CET  | root | timeline | timeline                                                  |                                                                 
single | 2451 |       | Fri 23 Dec 2016 10:00:05 PM CET  | root | timeline | timeline                                                  |                                                                 
single | 2452 |       | Fri 23 Dec 2016 10:00:10 PM CET  | root |          |                                                           | 

The problem with this is, that this is never cleaned up. So a tag like "backup_inprogress" would be a way to indicate failed backups. Next time snap-sync can detect this, warn and ask for deletion.

Edit:
Got config in use again after i try to transfer again:

Will backup //.snapshots/2474/snapshot to /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/2474//snapshot
Continue with backup [Y/n]? y
At subvol //.snapshots/2474/snapshot
Config is in use.

Feedback

Hi, I've just found your backup script which is now a fance package. Well done so far!
I've got some feedback which you might also apply to your software. If it gets enough votes on AUR I can also move this one to community. I'd love to do so ;)

  • No help message is available
  • The snapper configs to packup could be selectable via parameters: -c root home
  • or via a config setting in the snapper file itself. For example: snapper get-config SNAP-SYNC_EXCLUDE=yes
  • Readme typo: "Additionally you are show the location of the backed up snapshot."
  • "Enter name of directory to store backups:" should make more clear that this is a relative path and it does not need to exist: "Enter name of relative directory to store backups (created if not existing):"
  • Continue needs no "enter" only a "y". I personally do not like this behaviour.
  • A quiet -q option is missing.
  • Systemd timer for weekly/daily etc backups would be great.
  • An optional -d --description option could add additional description to the snapshot. For example snap-sync weekly with the timer above
  • Also copying the index file would be highly important to get the date of the snapshot
  • If you have multiple configurations and do the first time backup it prompts you for the path, backups the subvol and then asks for the path of the next config. All those setting should be askes first.

If you ask me in general I'd use the snapper config files itself for additional settings. I'd start with an initital config check, if new configuration (appart root etc) were found. Then snap-sync will configure those and start the backup afterwards. The quiet uption would skip the setting but possibly print a warning for a new (not backuped) config.

This tool is totally awesome and really essential for everyones snapper setup. Thanks a lot for sharing!

Newly created folder has 644 permissions

If you want to create the initital backup the new location has standard permissions. This mean everyone can read your snapshot data. On the local disk this is disabled by default. The newly created folder should be mkdir -p -m700 /path.

Execution bits on systemd services

As reported on AUR:

daftaupe commented on 2016-11-24 16:46

Configuration file /usr/lib/systemd/system/[email protected] is marked executable. Please remove executable permission bits. Proceeding anyway.
Configuration file /usr/lib/systemd/system/[email protected] is marked executable. Please remove executable permission bits. Proceeding anyway.

I get these 2 warnings from systemd in my journal, I guess that could be fixed. Thanks.

Tag new release

Its been a while and a lot of bugs have been fixed since the last release. Maybe you can tag another one?

Config is un use

[arch@arch ~]$ sudo snap-sync
Select a mounted BTRFS device to backup to.
   1) e95284fd-f67a-4d64-af68-1454db9b11fa (/)
   2) e95284fd-f67a-4d64-af68-1454db9b11fa (/home)
   3) e95284fd-f67a-4d64-af68-1454db9b11fa (/.snapshots)
   4) 4f1cabfc-cd6b-4012-8ec6-9073d84f2300 (/mnt2)
   5) 80acb842-b065-4a11-9b88-71619b25f48b (/mnt)
   0) Exit
Enter a number: 4

You selected the disk with UUID 4f1cabfc-cd6b-4012-8ec6-9073d84f2300.
The disk is mounted at /mnt2.

Will backup //.snapshots/117/snapshot to /mnt2/test/root/117//snapshot
Continue with backup [Y/n]? 
At subvol //.snapshots/117/snapshot
Config is in use.

I am not sure why this happens. Snapper list gives me those entries:

[arch@arch ~]$ sudo snapper list
Type   | #   | Pre # | Date                            | User | Cleanup  | Description                                                                                                                        | Userdata                                                 
-------+-----+-------+---------------------------------+------+----------+------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------
single | 0   |       |                                 | root |          | current                                                                                                                            |                                                          
single | 1   |       | Mon 28 Nov 2016 06:26:07 PM CET | root | timeline | timeline                                                                                                                           |                                                          
pre    | 4   |       | Mon 28 Nov 2016 06:37:01 PM CET | root | number   | pacman -S arch-install-scripts                                                                                                     |                                                          
post   | 5   | 4     | Mon 28 Nov 2016 06:37:02 PM CET | root | number   | pacman -S arch-install-scripts                                                                                                     |                                                          
single | 11  |       | Tue 29 Nov 2016 12:00:08 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 35  |       | Wed 30 Nov 2016 12:00:08 AM CET | root | timeline | timeline                                                                                                                           |                                                          
pre    | 40  |       | Fri 02 Dec 2016 08:17:10 PM CET | root | number   | pacman -S git devtools                                                                                                             |                                                          
post   | 41  | 40    | Fri 02 Dec 2016 08:17:14 PM CET | root | number   | pacman -S git devtools                                                                                                             |                                                          
pre    | 42  |       | Fri 02 Dec 2016 08:20:20 PM CET | root | number   | pacman -U cower-16-1-x86_64.pkg.tar.xz                                                                                             |                                                          
post   | 43  | 42    | Fri 02 Dec 2016 08:20:21 PM CET | root | number   | pacman -U cower-16-1-x86_64.pkg.tar.xz                                                                                             |                                                          
pre    | 44  |       | Fri 02 Dec 2016 08:24:28 PM CET | root | number   | pacman -U pacaur-4.6.10-1-any.pkg.tar.xz                                                                                           |                                                          
post   | 45  | 44    | Fri 02 Dec 2016 08:24:29 PM CET | root | number   | pacman -U pacaur-4.6.10-1-any.pkg.tar.xz                                                                                           |                                                          
pre    | 46  |       | Fri 02 Dec 2016 08:30:22 PM CET | root | number   | pacman -Syu                                                                                                                        |                                                          
post   | 47  | 46    | Fri 02 Dec 2016 08:31:22 PM CET | root | number   | pacman -Syu                                                                                                                        |                                                          
pre    | 48  |       | Fri 02 Dec 2016 08:37:00 PM CET | root | number   | pacman --color never --noconfirm -S --asdeps gnuplot gconf                                                                         |                                                          
post   | 49  | 48    | Fri 02 Dec 2016 08:37:08 PM CET | root | number   | pacman --color never --noconfirm -S --asdeps gnuplot gconf                                                                         |                                                          
pre    | 50  |       | Fri 02 Dec 2016 08:38:31 PM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/harmony-player/harmony-player-0.4.3-1-x86_64.pkg.tar.xz --ask 36 --noconfirm                   |                                                          
post   | 51  | 50    | Fri 02 Dec 2016 08:38:34 PM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/harmony-player/harmony-player-0.4.3-1-x86_64.pkg.tar.xz --ask 36 --noconfirm                   |                                                          
single | 52  |       | Fri 02 Dec 2016 09:00:03 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 56  |       | Sat 03 Dec 2016 12:00:03 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 64  |       | Sat 03 Dec 2016 08:00:03 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 65  |       | Sat 03 Dec 2016 09:00:03 AM CET | root | timeline | timeline                                                                                                                           |                                                          
pre    | 66  |       | Sat 03 Dec 2016 09:46:46 AM CET | root | number   | pacman -Syu                                                                                                                        |                                                          
post   | 67  | 66    | Sat 03 Dec 2016 09:46:48 AM CET | root | number   | pacman -Syu                                                                                                                        |                                                          
single | 68  |       | Sat 03 Dec 2016 10:00:08 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 69  |       | Sat 03 Dec 2016 11:00:08 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 70  |       | Sat 03 Dec 2016 12:00:08 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 71  |       | Sat 03 Dec 2016 01:00:08 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 72  |       | Sat 03 Dec 2016 02:00:08 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 73  |       | Sat 03 Dec 2016 03:00:08 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 74  |       | Sat 03 Dec 2016 04:00:08 PM CET | root | timeline | timeline                                                                                                                           |                                                          
pre    | 75  |       | Sun 04 Dec 2016 01:13:58 AM CET | root | number   | pacman -U /home/arch/snap-sync/snap-sync-0.2.2-1-any.pkg.tar.xz                                                                    |                                                          
post   | 76  | 75    | Sun 04 Dec 2016 01:13:58 AM CET | root | number   | pacman -U /home/arch/snap-sync/snap-sync-0.2.2-1-any.pkg.tar.xz                                                                    |                                                          
single | 77  |       | Sun 04 Dec 2016 01:15:34 AM CET | root |          | latest incremental backup                                                                                                          | backupdir=test, uuid=4f1cabfc-cd6b-4012-8ec6-9073d84f2300
single | 78  |       | Sun 04 Dec 2016 02:00:26 AM CET | root | timeline | timeline                                                                                                                           |                                                          
pre    | 79  |       | Sun 04 Dec 2016 02:18:44 AM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/xfce4-volumed/xfce4-volumed-0.1.13-7-x86_64.pkg.tar.xz --ask 36 --noconfirm                    |                                                          
post   | 80  | 79    | Sun 04 Dec 2016 02:18:46 AM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/xfce4-volumed/xfce4-volumed-0.1.13-7-x86_64.pkg.tar.xz --ask 36 --noconfirm                    |                                                          
pre    | 81  |       | Sun 04 Dec 2016 02:19:10 AM CET | root | number   | pacman --color never --noconfirm -S --asdeps libkeybinder3                                                                         |                                                          
post   | 82  | 81    | Sun 04 Dec 2016 02:19:10 AM CET | root | number   | pacman --color never --noconfirm -S --asdeps libkeybinder3                                                                         |                                                          
pre    | 83  |       | Sun 04 Dec 2016 02:19:11 AM CET | root | number   | pacman --color never --noconfirm -S --asdeps intltool                                                                              |                                                          
post   | 84  | 83    | Sun 04 Dec 2016 02:19:11 AM CET | root | number   | pacman --color never --noconfirm -S --asdeps intltool                                                                              |                                                          
pre    | 85  |       | Sun 04 Dec 2016 02:19:20 AM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/xfce4-pulseaudio-plugin/xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz --ask 36 --noconfirm |                                                          
post   | 86  | 85    | Sun 04 Dec 2016 02:19:22 AM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/xfce4-pulseaudio-plugin/xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz --ask 36 --noconfirm |                                                          
pre    | 87  |       | Sun 04 Dec 2016 02:20:51 AM CET | root | number   | pacman -Rs xfce4-volumed                                                                                                           |                                                          
post   | 88  | 87    | Sun 04 Dec 2016 02:20:51 AM CET | root | number   | pacman -Rs xfce4-volumed                                                                                                           |                                                          
pre    | 89  |       | Sun 04 Dec 2016 02:25:39 AM CET | root | number   | pacman -S xfce4-notifyd                                                                                                            |                                                          
post   | 90  | 89    | Sun 04 Dec 2016 02:25:39 AM CET | root | number   | pacman -S xfce4-notifyd                                                                                                            |                                                          
pre    | 91  |       | Sun 04 Dec 2016 02:27:43 AM CET | root | number   | pacman -Rs xfce4-pulseaudio-plugin                                                                                                 |                                                          
post   | 92  | 91    | Sun 04 Dec 2016 02:27:44 AM CET | root | number   | pacman -Rs xfce4-pulseaudio-plugin                                                                                                 |                                                          
pre    | 93  |       | Sun 04 Dec 2016 02:28:08 AM CET | root | number   | pacman --color never --noconfirm -S --asdeps libkeybinder3                                                                         |                                                          
post   | 94  | 93    | Sun 04 Dec 2016 02:28:08 AM CET | root | number   | pacman --color never --noconfirm -S --asdeps libkeybinder3                                                                         |                                                          
pre    | 95  |       | Sun 04 Dec 2016 02:28:17 AM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/xfce4-pulseaudio-plugin/xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz --ask 36 --noconfirm |                                                          
post   | 96  | 95    | Sun 04 Dec 2016 02:28:17 AM CET | root | number   | pacman -Ud /home/arch/.cache/pacaur/xfce4-pulseaudio-plugin/xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz --ask 36 --noconfirm |                                                          
pre    | 97  |       | Sun 04 Dec 2016 02:30:35 AM CET | root | number   | pacman -U xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz                                                                        |                                                          
post   | 98  | 97    | Sun 04 Dec 2016 02:30:35 AM CET | root | number   | pacman -U xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz                                                                        |                                                          
single | 99  |       | Sun 04 Dec 2016 03:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 100 |       | Sun 04 Dec 2016 04:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 101 |       | Sun 04 Dec 2016 05:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 102 |       | Sun 04 Dec 2016 06:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 103 |       | Sun 04 Dec 2016 07:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 104 |       | Sun 04 Dec 2016 08:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 105 |       | Sun 04 Dec 2016 09:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 106 |       | Sun 04 Dec 2016 10:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 107 |       | Sun 04 Dec 2016 11:00:24 AM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 108 |       | Sun 04 Dec 2016 12:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 109 |       | Sun 04 Dec 2016 01:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 110 |       | Sun 04 Dec 2016 02:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 111 |       | Sun 04 Dec 2016 03:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 112 |       | Sun 04 Dec 2016 04:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 113 |       | Sun 04 Dec 2016 05:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 114 |       | Sun 04 Dec 2016 06:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 115 |       | Sun 04 Dec 2016 07:00:24 PM CET | root | timeline | timeline                                                                                                                           |                                                          
single | 116 |       | Sun 04 Dec 2016 07:01:03 PM CET | root |          |                                                                                                                                    |                                                          
single | 117 |       | Sun 04 Dec 2016 07:01:19 PM CET | root |          |                                                                                                                                    |                                                          
[arch@arch ~]$ 

I think moved the data a bit. The first mountpoint was somehwere at /run/media/name and now is mnt2. Before the folder on the partition was test and now is the root (i moved the data with mv).

This might have caused trouble:

[arch@arch ~]$ sudo ls /mnt2/test/
root
[arch@arch ~]$ sudo ls /mnt2/test/root
116  117  118
[arch@arch ~]$ sudo ls /mnt2/
home		     test						 xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz.asc
PKGBUILD	     xfce4-pulseaudio-plugin-0.2.4-1-x86_64-build.log	 xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz-namcap.log
PKGBUILD-namcap.log  xfce4-pulseaudio-plugin-0.2.4-1-x86_64-package.log  xfce4-pulseaudio-plugin-0.2.4.tar.bz2
root		     xfce4-pulseaudio-plugin-0.2.4-1-x86_64.pkg.tar.xz	 xfce4-pulseaudio-plugin.install

Notifications

It would be nice to get a notification when the service wants to do a backup. Also it would be nice to add a notification of the backup finished. I've seen examples in bach which are extremely simpler to integrate into libnotify. I can imagine notifications like this:

  • "Backing up configuration xyz"
  • "Please mount partition xyz for snap-sync backup"
  • "Backup of configuration xyz finished"

Default option does not accept "enter"

Will backup //.snapshots/5/snapshot to /mnt/////backup/root/5//snapshot
Continue with backup [Y/n]? 
Aborting backup for this configuration.

If you press enter I expect the program to use the upper case option (yes). It aborts.

ERROR: unable to resolve -c

[arch@arch ~]$ sudo snap-sync -n
[sudo] password for arch: 
Select a mounted BTRFS device to backup to.
   1) c91384af-f210-48f5-ba19-d6415a1bb447 (/)
   2) c91384af-f210-48f5-ba19-d6415a1bb447 (/hackallthethings)
   3) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/tmp)
   4) c91384af-f210-48f5-ba19-d6415a1bb447 (/data)
   5) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/abs)
   6) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/log)
   7) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/cache/pacman/pkg)
   8) c91384af-f210-48f5-ba19-d6415a1bb447 (/home)
   9) c91384af-f210-48f5-ba19-d6415a1bb447 (/.snapshots)
  10) c91384af-f210-48f5-ba19-d6415a1bb447 (/srv)
  11) c91384af-f210-48f5-ba19-d6415a1bb447 (/home/arch/hackallthethings)
  12) c91384af-f210-48f5-ba19-d6415a1bb447 (/home/arch/.local/share/Steam)
  13) e5b8a2d6-85a1-4640-a659-acb8d6922bd6 (/run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6)
   0) Exit
Enter a number: 13

You selected the disk with UUID e5b8a2d6-85a1-4640-a659-acb8d6922bd6.
The disk is mounted at /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6.

Creating new snapshot for root...
Will backup //.snapshots/4641/snapshot to /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/4641//snapshot
Sending incremental snapshot for root...
ERROR: unable to resolve -c

No idea why this happens..

Without -n:

Creating new snapshot for root...
Will backup //.snapshots/4642/snapshot to /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/4642//snapshot
Continue with backup [Y/n]? y
Sending incremental snapshot for root...
ERROR: unable to resolve -c
[arch@arch ~]$ snapper list | grep -e snap -e backup
single | 4465 |       | Fri 12 May 2017 01:48:56 PM CEST | root |          | latest incremental backup                           | backupdir=Backup/E744, uuid=d2133005-da15-484f-9dbc-3352e6af0529
single | 4514 |       | Sun 14 May 2017 07:19:17 PM CEST | root |          | latest incremental backup                           | backupdir=backup/E744, uuid=e5b8a2d6-85a1-4640-a659-acb8d6922bd6
single | 4641 |       | Tue 23 May 2017 07:20:10 PM CEST | root |          | snap-sync backup in progress                        |                                                                 
single | 4642 |       | Tue 23 May 2017 07:21:08 PM CEST | root |          | snap-sync backup in progress                        |      
+ printf 'Will backup %s to %s\n' //.snapshots/4644/snapshot /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/4644//snapshot
+ tee /tmp/tmp.V6AaRo9lXE/snap-sync.out
Will backup //.snapshots/4644/snapshot to /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/4644//snapshot
+ cont_backup=K
+ [[ no == \y\e\s ]]
+ [[ -n K ]]
+ [[ K != [Yy]\e\s ]]
+ [[ K != [Yy] ]]
+ [[ K != [Nn]\o ]]
+ [[ K != [Nn] ]]
+ read -r -p 'Continue with backup [Y/n]? ' cont_backup
Continue with backup [Y/n]? 
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ '' != [Yy]\e\s ]]
+ [[ '' != [Yy] ]]
+ [[ -n '' ]]
+ mkdir -p /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/4644/
+ [[ -z 4514 ]]
+ tee /tmp/tmp.V6AaRo9lXE/snap-sync.out
+ printf 'Sending incremental snapshot for %s...\n' root
Sending incremental snapshot for root...
+ btrfs send //.snapshots/4644/snapshot -c //.snapshots/4514/snapshot
+ btrfs receive /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/4644/
ERROR: unable to resolve -c
[arch@arch ~]$ sudo btrfs subvolume list / | grep 4514
ID 11975 gen 352561 top level 259 path @snapshots/4514/snapshot
[arch@arch ~]$ ls //.snapshots/4514/snapshot
bin  boot  data  dev  etc  hackallthethings  home  hostrun  lib  lib64  lost+found  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
[arch@arch ~]$ sudo btrfs sub list -auR / | grep 4514
ID 11975 gen 352561 top level 259 received_uuid - uuid d22e8c45-76cd-1e4f-bb1c-5dc37a51e313 path <FS_TREE>/@snapshots/4514/snapshot
ID 12054 gen 354514 top level 259 received_uuid - uuid 56fc424b-5d24-0c4c-8e51-c8467c4aaebf path <FS_TREE>/@snapshots/4548/snapshot
[arch@arch ~]$ sudo btrfs sub list -auR /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/ | grep 4514
ID 1557 gen 687 top level 5 received_uuid d22e8c45-76cd-1e4f-bb1c-5dc37a51e313 uuid 4904aefc-6884-e242-90c5-f70eb3b70f49 path backup/E744/root/4514/snapshot

18 commits to master since the last release. Maybe this fixed it already?

Create manpage

Offload most of the README to the manpage except for a few examples.

No linefeed after config error

[alarm@alarmpi snap-sync]$ sudo snap-sync -c k
Select a mounted BTRFS device to backup to.
1) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/)
2) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/tmp)
3) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/log)
4) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/abs)
5) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/.snapshots)
6) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/home)
7) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/data)
8) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/srv)
9) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/cache/pacman/pkg)
10) 6c1a3286-34c5-4bbd-a7cc-5762c8dd5ced (/mnt)
0) Exit
Enter a number: 10

You selected the disk with UUID 6c1a3286-34c5-4bbd-a7cc-5762c8dd5ced.
The disk is mounted at /mnt.

Error: Selected snapper configuration k does not exist.[alarm@alarmpi snap-sync]$ 

Also it makes sense to check the configs first before you select the backup location.

[FR] Ignore configs with special setting

Currently you have the option -c root home to include special configs. However it would be nice to exclude some via:
snapper set-config SNAP-SYNC_EXCLUDE=yes

This is especially important for the systemd timer, as by default it backs up every config. and editing services is not the way to do. This way you could exclude settings via snappers internal config files.

snapshot failed: File exists

snap-sync 0.4.2

$ sudo snap-sync 
Select a mounted BTRFS device on your local machine to backup to.
   1) e5b8a2d6-85a1-4640-a659-acb8d6922bd6 (/run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6)
   0) Exit
Enter a number: 1

You selected the disk with UUID e5b8a2d6-85a1-4640-a659-acb8d6922bd6.
The disk is mounted at /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6.

Initial configuration...

NOTE: Previous failed snap-sync backup snapshots found for 'root'.
Delete failed backup snapshots [y/N]? y

Creating new snapshot for root...
Will backup //.snapshots/6356/snapshot to /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/root/6356//snapshot
Continue with backup [Y/n]? y

NOTE: Previous failed snap-sync backup snapshots found for 'home'.
Delete failed backup snapshots [y/N]? y

Creating new snapshot for home...
Will backup /home/.snapshots/4315/snapshot to /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/home/4315//snapshot
Continue with backup [Y/n]? y

NOTE: Previous failed snap-sync backup snapshots found for 'hackallthethings'.
Delete failed backup snapshots [y/N]? y

Creating new snapshot for hackallthethings...
yWill backup /hackallthethings/.snapshots/3732/snapshot to /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744/hackallthethings/3732//snapshot
Continue with backup [Y/n]? 

Performing backups...

Sending incremental snapshot for root...
At subvol //.snapshots/6356/snapshot
At snapshot snapshot
ERROR: creating snapshot backup/E744/root/5758/snapshot -> snapshot failed: File exists
Exited due to error on line 404.
exit status: 1
command: $ssh btrfs receive "$backup_location"
bash line: 0
function name: main
$ snapper list
Type   | #    | Pre # | Date                             | User | Cleanup  | Description                                  | Userdata                                                         
-------+------+-------+----------------------------------+------+----------+----------------------------------------------+------------------------------------------------------------------
single | 0    |       |                                  | root |          | current                                      |                                                                  
single | 5749 |       | Sat 05 Aug 2017 09:35:25 PM CEST | root |          | latest incremental backup                    | backupdir=Backup/E744/, uuid=d2133005-da15-484f-9dbc-3352e6af0529
single | 5758 |       | Sun 06 Aug 2017 07:35:11 AM CEST | root |          | latest incremental backup                    | backupdir=backup/E744, uuid=e5b8a2d6-85a1-4640-a659-acb8d6922bd6 

GPG signatures for source validation

As we all know, today more than ever before, it is crucial to be able to trust our computing environments. One of the main difficulties that package maintainers of Linux distributions face, is the difficulty to verify the authenticity and the integrity of the source code.

The Arch Linux team would appreciate it if you would provide us GPG signatures in order to verify easily and quickly of your source code releases.

Overview of the required tasks:

Additional Information:

Thanks.

Abort on wrong user input

You selected the disk with UUID e9155055-f7e6-4975-90c5-3994057b8c64.
The disk is mounted at /media/Raid.

No backups have been performed for 'root' on this disk.
Enter name of directory to store backups, relative to /media/Raid (to be created if not existing): backup/rpi
This will be the initial backup for snapper configuration 'root' to this disk. This could take awhile.
Will backup //.snapshots/8/snapshot to /media/Raid/backup/rpi/root/8//snapshot
Continue with backup [Y/n]? k
Aborting backup for this configuration.

No backups have been performed for 'home' on this disk.
Enter name of directory to store backups, relative to /media/Raid (to be created if not existing): 

If you trype something invalid it should give an error and try again. Too bad "k" is not interpreted as "y" (joke).

Exclude source btrfs subvolumes

It would be nice to exclude the subvolumes that the system is installed on, so you get less entries. For me those are by default min 11 which could be reduced. You can identify them by uuid I think.

Use upstream's location for configuration file

Other distributions follow upstream's location of /etc/sysconfig/snapper. Also add an installation flag of some sort to change this location (e.g., for Arch it would be /etc/conf.d/snapper). Then for Arch the flag could be used with the AUR package.

cf. #42

Does not stop on invalid uuid

[alarm@alarmpi snap-sync]$ sudo snap-sync -u hackallthethings -n
A device with UUID hackallthethings was not found to be mounted, or it is not a BTRFS device.
Select a mounted BTRFS device to backup to.
1) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/)
2) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/tmp)
3) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/log)
4) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/abs)
5) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/.snapshots)
6) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/home)
7) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/data)
8) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/srv)
9) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/cache/pacman/pkg)
10) 6c1a3286-34c5-4bbd-a7cc-5762c8dd5ced (/mnt)
0) Exit
Enter a number: 10

You selected the disk with UUID 6c1a3286-34c5-4bbd-a7cc-5762c8dd5ced.
The disk is mounted at /mnt.

Will backup //.snapshots/11/snapshot to /mnt/////backup/root/11//snapshot
At subvol //.snapshots/11/snapshot

Will backup /home/.snapshots/8/snapshot to /mnt/////backup/home/8//snapshot
At subvol /home/.snapshots/8/snapshot

Done!

GPG key expired

Your key expired as you can see in this commit:
01b7237

You need to update it, publish it to the keyservers and possibly update the github profile. But i would not edit the github profile unless you need to. Otherwise it could make every commit so far unverified again (permanently). At least thats what i remember from the docs.

Does not work with multiple subvolumes on one disk

I found two problems when running snap-sync on a server. The first is a simple issue: notify-send seems to require a desktop GUI. Headless servers do not have the ability to install notify-send afaik. I modified snap-sync to send messages to the console.

The second issue is that when a disk contains multiple subvolumes and each of these has a snapper config, snap-sync was not putting each snapshot into its own appropriate subdirectory. The snapshots with the same numbers were colliding. I fixed this by making snap-sync pick the backup subdirectory location from the name of the snapper config.

Here is my modified snap-sync script. There are only a few changes.

#!/bin/bash
# James W. Barnett
# 2017.07.09 modified by DavesTechShop

# Takes snapshots of each snapper configuration. It then sends the snapshot to
# a location on an external drive. After the initial transfer, it does
# incremental snapshots on later calls. It's important not to delete the
# snapshot created on your system since that will be used to determine the
# difference for the next incremental snapshot.

version="0.3.1"
name="snap-sync"

TMPDIR=$(mktemp -d)
PIPE=$TMPDIR/$name.out
mkfifo $PIPE
systemd-cat -t "$name" < $PIPE &
exec 3>$PIPE

notify_info() {
    #for u in $(users); do
    #    sudo -u $u DISPLAY=:0 \
    #    DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(sudo -u $u id -u)/bus \
    #    notify-send -a $name "$1" "$2" --icon=dialog-information
    #done
    echo INFORMATION "$1" "$2"
}

notify_error() {
    #for u in $(users); do
    #    sudo -u $u DISPLAY=:0 \
    #    DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(sudo -u $u id -u)/bus \
    #    notify-send -a $name "$1" "$2" --icon=dialog-error
    #done
    echo ERROR "$1" "$2"
}

out() { 
    printf "$1 $2\n" "${@:3}" | tee $PIPE
}

error() { 
    out "==> ERROR:" "$@"
    notify_error 'Error' 'Check journal for more information.'
} >&2

die() { 
    error "$@"
    exit 1
}

sigkill() { 
    die "Exited due to user intervention." 
}

[[ $EUID -ne 0 ]] && die "Script must be run as root." 

usage() {
cat <<EOF
$name $version
Usage: $name [options]

Options:
-d, --description <desc> Change the snapper description. Default: "latest incremental backup"
-c, --config <config>    Specify the snapper configuration to use. Otherwise will perform for each snapper
                        configuration. Can list multiple configurations within quotes, space-separated
                        (e.g. -c "root home").
-n, --noconfirm          Do not ask for confirmation for each configuration. Will still prompt for backup
                        directory name on first backup"
-u, --UUID <UUID>        Specify the UUID of the mounted BTRFS subvolume to back up to. Otherwise will prompt."
                        If multiple mount points are found with the same UUID, will prompt user."
--remote <address>       Send the snapshot backup to a remote machine. The snapshot will be sent via ssh. You 
                        should specify the remote machine's hostname or ip address. The 'root' user must be 
                        permitted to login on the remote machine.

EOF
}

trap error ERR
trap sigkill SIGTERM SIGINT
ssh=""

while [[ $# -gt 0 ]]; do
    key="$1"
    case $key in
        -d|--description)
            description="$2"
            shift 2
        ;;
        -c|--config)
            selected_configs="$2"
            shift 2
        ;;
        -u|--UUID)
            uuid_cmdline="$2"
            shift 2
        ;;
        -n|--noconfirm)
            noconfirm="yes"
            shift
        ;;
        -h|--help)
            usage
            exit 1
        ;;
            --remote)
            remote=$2
            ssh="ssh $remote"
            shift 2
            ;;
        *)
            die "Unknown option: $key\nRun '$name -h' for valid options.\n"
        ;;
    esac
done

description=${description:-"latest incremental backup"}
uuid_cmdline=${uuid_cmdline:-"none"}
noconfirm=${noconfirm:-"no"}

if [[ "$uuid_cmdline" != "none" ]]; then
    if [[ -z $ssh ]]; then
        notify_info "Backup started" "Starting backups to $uuid_cmdline..."
    else
        notify_info "Backup started" "Starting backups to $uuid_cmdline at $remote..."
    fi
else
    if [[ -z $ssh ]]; then
        notify_info "Backup started" "Starting backups. Use command line menu to select disk."
    else
        notify_info "Backup started" "Starting backups. Use command line menu to select disk on $remote."
    fi
fi

TARGETS="$($ssh findmnt -n -v -t btrfs -o TARGET --list)"
UUIDS="$($ssh findmnt -n -v -t btrfs -o UUID --list)"

declare -a TARGETS_ARRAY
declare -a UUIDS_ARRAY

i=0
disk=-1
disk_count=0
for x in $UUIDS; do
    UUIDS_ARRAY[$i]=$x
    if [[ "$x" == "$uuid_cmdline" ]]; then
        disk=$i
        disk_count=$(($disk_count+1))
    fi
    i=$((i+1))
done

i=0
for x in $TARGETS; do
    TARGETS_ARRAY[$i]=$x
    i=$((i+1))
done

if [[ "$disk_count" > 1 ]]; then
    printf "Multiple mount points were found with UUID $uuid_cmdline.\n"
    disk="-1"
fi

if [[ "$disk" == -1 ]]; then
    if [[ "$disk_count" == 0 && "$uuid_cmdline" != "none" ]]; then
        error "A device with UUID $uuid_cmdline was not found to be mounted, or it is not a BTRFS device."
    fi
    if [[ -z $ssh ]]; then
        printf "Select a mounted BTRFS device on your local machine to backup to.\n"
    else
        printf "Select a mounted BTRFS device on $remote to backup to.\n"
    fi
    while [[ $disk -lt 0 || $disk -gt $i ]]; do
        for x in "${!TARGETS_ARRAY[@]}"; do
            printf "%4s) %s (%s)\n" "$((x+1))" "${UUIDS_ARRAY[$x]}" "${TARGETS_ARRAY[$x]}"
        done
        printf "%4s) Exit\n" "0"
        read -r -p "Enter a number: " disk
        if ! [[ $disk == ?(-)+([0-9]) ]]; then
            printf "\nNo disk selected. Select a disk to continue.\n"
            disk=-1
        fi
    done
    if [[ $disk == 0 ]]; then
        exit 0
    fi
    disk=$(($disk-1))
fi

selected_uuid="${UUIDS_ARRAY[$((disk))]}"
selected_mnt="${TARGETS_ARRAY[$((disk))]}"
printf "\nYou selected the disk with UUID %s.\n" "$selected_uuid" | tee $PIPE
if [[ -z $ssh ]]; then
    printf "The disk is mounted at %s.\n" "$selected_mnt" | tee $PIPE
else
    printf "The disk is mounted at %s:%s.\n" "$remote" "$selected_mnt" | tee $PIPE
fi

if [[ -f /etc/conf.d/snapper ]]; then
    source /etc/conf.d/snapper 
else
    die "/etc/conf.d/snapper does not exist!"
fi

selected_configs=${selected_configs:-$SNAPPER_CONFIGS}

declare -a BACKUPDIRS_ARRAY
declare -a MYBACKUPDIR_ARRAY
declare -a OLD_NUM_ARRAY
declare -a OLD_SNAP_ARRAY
declare -a NEW_NUM_ARRAY
declare -a NEW_SNAP_ARRAY
declare -a NEW_INFO_ARRAY
declare -a BACKUPLOC_ARRAY
declare -a CONT_BACKUP_ARRAY

printf "\nInitial configuration...\n" | tee $PIPE

# Initial configuration of where backup directories are
i=0
for x in $selected_configs; do

    if [[ "$(snapper -c $x list -t single | awk '/'$name' backup in progress/ {cnt++} END {print cnt}')" -gt 0 ]]; then
        printf "\nNOTE: Previous failed $name backup snapshots found for '$x'.\n" | tee $PIPE
    fi

    SNAP_SYNC_EXCLUDE=no
    unset BACKUPDIR

    if [[ -f "/etc/snapper/configs/$x" ]]; then
        source /etc/snapper/configs/$x
    else
        die "Selected snapper configuration $x does not exist."
    fi

    if [[ $SNAP_SYNC_EXCLUDE == "yes" ]]; then 
        continue
    fi

    printf "\n"

    old_num=$(snapper -c "$x" list -t single | awk '/'"$selected_uuid"'/ {print $1}')
    old_snap=$SUBVOLUME/.snapshots/$old_num/snapshot

    OLD_NUM_ARRAY[$i]=$old_num
    OLD_SNAP_ARRAY[$i]=$old_snap

    if [[ -z "$old_num" ]]; then
        printf "No backups have been performed for '%s' on this disk.\n" "$x"
        #read -r -p "Enter name of directory to store backups, relative to $selected_mnt (to be created if not existing): " mybackupdir
        mybackupdir="$x"
        printf "This will be the initial backup for snapper configuration '%s' to this disk. This could take awhile.\n" "$x"
        BACKUPDIR="$selected_mnt/$mybackupdir"
        $ssh mkdir -p -m700 "$BACKUPDIR"
    else
        mybackupdir=$(snapper -c "$x" list -t single | awk -F"|" '/'"$selected_uuid"'/ {print $5}' | awk -F "," '/backupdir/ {print $1}' | awk -F"=" '{print $2}')
        BACKUPDIR="$selected_mnt/$mybackupdir"
        $ssh test -d $BACKUPDIR || die "%s is not a directory on %s.\n" "$BACKUPDIR" "$selected_uuid"
    fi
    BACKUPDIRS_ARRAY[$i]="$BACKUPDIR"
    MYBACKUPDIR_ARRAY[$i]="$mybackupdir"

    printf "Creating new snapshot for $x...\n" | tee $PIPE
    new_num=$(snapper -c "$x" create --print-number -d "$name backup in progress")
    new_snap=$SUBVOLUME/.snapshots/$new_num/snapshot
    new_info=$SUBVOLUME/.snapshots/$new_num/info.xml
    sync
    #backup_location=$BACKUPDIR/$x/$new_num/
    backup_location=$BACKUPDIR/$new_num/
    if [[ -z $ssh ]]; then
        printf "Will backup %s to %s\n" "$new_snap" "$backup_location/snapshot" | tee $PIPE
    else
        printf "Will backup %s to %s\n" "$new_snap" "$remote":"$backup_location/snapshot" | tee $PIPE
    fi

    NEW_NUM_ARRAY[$i]="$new_num"
    NEW_SNAP_ARRAY[$i]="$new_snap"
    NEW_INFO_ARRAY[$i]="$new_info"
    BACKUPLOC_ARRAY[$i]="$backup_location"

    cont_backup="K"
    CONT_BACKUP_ARRAY[$i]="yes"
    if [[ $noconfirm == "yes" ]]; then
        cont_backup="yes"
    else
        while [[ -n "$cont_backup" && "$cont_backup" != [Yy]"es" &&
            "$cont_backup" != [Yy] && "$cont_backup" != [Nn]"o" &&
            "$cont_backup" != [Nn] ]]; do
            read -r -p "Continue with backup [Y/n]? " cont_backup
            if [[ -n "$cont_backup" && "$cont_backup" != [Yy]"es" &&
            "$cont_backup" != [Yy] && "$cont_backup" != [Nn]"o" &&
            "$cont_backup" != [Nn] ]]; then
                printf "Select 'Y' or 'n'.\n"
            fi
        done
    fi

    if [[ "$cont_backup" != [Yy]"es" && "$cont_backup" != [Yy] && -n "$cont_backup" ]]; then
        CONT_BACKUP_ARRAY[$i]="no"
        printf "Aborting backup for this configuration.\n"
        snapper -c $x delete $new_num
    fi

    i=$(($i+1))

done

# Actual backing up
printf "\nPerforming backups...\n" | tee $PIPE
i=-1
for x in $selected_configs; do

    i=$(($i+1))

    SNAP_SYNC_EXCLUDE=no

    if [[ -f "/etc/snapper/configs/$x" ]]; then
        source /etc/snapper/configs/$x
    else
        die "Selected snapper configuration $x does not exist."
    fi

    cont_backup=${CONT_BACKUP_ARRAY[$i]}
    if [[ $cont_backup == "no" || $SNAP_SYNC_EXCLUDE == "yes" ]]; then 
        notify_info "Backup in progress" "NOTE: Skipping $x configuration."
        continue
    fi

    notify_info "Backup in progress" "Backing up $x configuration."

    printf "\n"

    old_num="${OLD_NUM_ARRAY[$i]}"
    old_snap="${OLD_SNAP_ARRAY[$i]}"
    BACKUPDIR="${BACKUPDIRS_ARRAY[$i]}"
    mybackupdir="${MYBACKUPDIR_ARRAY[$i]}"
    new_num="${NEW_NUM_ARRAY[$i]}"
    new_snap="${NEW_SNAP_ARRAY[$i]}"
    new_info="${NEW_INFO_ARRAY[$i]}"
    backup_location="${BACKUPLOC_ARRAY[$i]}"

    $ssh mkdir -p $backup_location

    if [[ -z "$old_num" ]]; then
        printf "Sending first snapshot for %s...\n" "$x" | tee $PIPE  
        btrfs send "$new_snap" | $ssh btrfs receive "$backup_location" &>/dev/null

    else

        printf "Sending incremental snapshot for %s...\n" "$x" | tee $PIPE  
        # Sends the difference between the new snapshot and old snapshot to the
        # backup location. Using the -c flag instead of -p tells it that there
        # is an identical subvolume to the old snapshot at the receiving
        # location where it can get its data. This helps speed up the transfer.
        btrfs send -c "$old_snap" "$new_snap" | $ssh btrfs receive "$backup_location"
        printf "Deleting old snapshot for $x...\n" | tee $PIPE
        snapper -c "$x" delete "$old_num"
    fi

    if [[ -z $ssh ]]; then
        cp "$new_info" "$backup_location"
    else
        rsync -avzq "$new_info" "$remote":"$backup_location"
    fi

    # It's important not to change this userdata in the snapshots, since that's how
    # we find the previous one.

    userdata="backupdir=$mybackupdir, uuid=$selected_uuid"

    # Tag new snapshot as the latest
    printf "Tagging new snapshot as latest backup for $x...\n" | tee $PIPE
    snapper -v -c "$x" modify -d "$description" -u "$userdata" "$new_num"

    printf "Backup complete for configuration %s.\n" "$x" > $PIPE

done

printf "\nDone!\n" | tee $PIPE
exec 3>&-

if [[ "$uuid_cmdline" != "none" ]]; then
    notify_info "Finished" "Backups to $uuid_cmdline complete!"
else
    notify_info "Finished" "Backups complete!"
fi

shellcheck linting

I use shellcheck myself for all of my scripts, it helps avoid common bash mistakes, I also have it integrated with Travis for my projects. @wesbarnett would you accept a PR with shellcheck changes integrated and a .travis.yml to have it checked?

Make configurations select-able

Most likely via command line option, but possibly in snapper configuration file. Will need to figure out which one overrides the other if both. See #1.

Add option to verify backup

Wouldnt it be possible to cmp both subvolumes after the sync? If btrfs send outputs the exact same output for src and dest the backup is okay. However I am not sure if btrfs send always work the same (same order of files). If not maybe there is another way to verify the backup?

device selection indention

[alarm@alarmpi snap-sync]$ sudo snap-sync -n
Select a mounted BTRFS device to backup to.
1) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/)
2) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/tmp)
3) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/log)
4) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/abs)
5) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/.snapshots)
6) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/home)
7) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/data)
8) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/srv)
9) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/cache/pacman/pkg)
10) 6c1a3286-34c5-4bbd-a7cc-5762c8dd5ced (/mnt)
0) Exit

device 10 is indented wrong. Looks not ocd ;)

binary operator expected

With the new release I got a new error now on 2nd run:

[arch@arch snap-sync]$ sudo snap-sync 
[sudo] password for arch: 
Select a mounted BTRFS device on your local machine to backup to.
   1) c91384af-f210-48f5-ba19-d6415a1bb447 (/)
   2) c91384af-f210-48f5-ba19-d6415a1bb447 (/home)
   3) c91384af-f210-48f5-ba19-d6415a1bb447 (/data)
   4) c91384af-f210-48f5-ba19-d6415a1bb447 (/.snapshots)
   5) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/cache/pacman/pkg)
   6) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/tmp)
   7) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/abs)
   8) c91384af-f210-48f5-ba19-d6415a1bb447 (/srv)
   9) c91384af-f210-48f5-ba19-d6415a1bb447 (/home/arch/hackallthethings)
  10) c91384af-f210-48f5-ba19-d6415a1bb447 (/hackallthethings)
  11) c91384af-f210-48f5-ba19-d6415a1bb447 (/var/log)
  12) c91384af-f210-48f5-ba19-d6415a1bb447 (/home/arch/.local/share/Steam)
  13) e5b8a2d6-85a1-4640-a659-acb8d6922bd6 (/run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6)
   0) Exit
Enter a number: 13

You selected the disk with UUID e5b8a2d6-85a1-4640-a659-acb8d6922bd6.
The disk is mounted at /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6.

Initial configuration...

NOTE: Previous failed snap-sync backup snapshots found for 'root'.

/usr/bin/snap-sync: line 253: test: /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744: binary operator expected
==> ERROR: /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744
backup/E744 is not a directory on e5b8a2d6-85a1-4640-a659-acb8d6922bd6.
$ ls /run/media/arch/e5b8a2d6-85a1-4640-a659-acb8d6922bd6/backup/E744
hackallthethings  home  root

issue when saving to backup root /

[alarm@alarmpi snap-sync]$ sudo snap-sync -n
Select a mounted BTRFS device to backup to.
1) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/)
2) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/tmp)
3) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/log)
4) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/abs)
5) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/.snapshots)
6) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/home)
7) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/data)
8) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/srv)
9) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/cache/pacman/pkg)
10) 1e44de19-7755-4219-9884-bc932958ac9e (/mnt)
0) Exit
Enter a number: 10

You selected the disk with UUID 1e44de19-7755-4219-9884-bc932958ac9e.
The disk is mounted at /mnt.

No backups have been performed for 'root' on this disk.
Enter name of directory to store backups, relative to /mnt (to be created if not existing):       
This will be the initial backup for snapper configuration 'root' to this disk. This could take awhile.
Will backup //.snapshots/12/snapshot to /mnt//root/12//snapshot
At subvol //.snapshots/12/snapshot

No backups have been performed for 'home' on this disk.
Enter name of directory to store backups, relative to /mnt (to be created if not existing): 
This will be the initial backup for snapper configuration 'home' to this disk. This could take awhile.
Will backup /home/.snapshots/9/snapshot to /mnt//home/9//snapshot
At subvol /home/.snapshots/9/snapshot

Done!
[alarm@alarmpi snap-sync]$ sudo snap-sync -n
[sudo] password for alarm: 
Select a mounted BTRFS device to backup to.
1) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/)
2) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/tmp)
3) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/log)
4) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/abs)
5) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/.snapshots)
6) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/home)
7) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/data)
8) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/srv)
9) 16bfd981-0f5e-4094-b1fb-da0579b560fa (/var/cache/pacman/pkg)
10) 1e44de19-7755-4219-9884-bc932958ac9e (/mnt)
0) Exit
Enter a number: 10

You selected the disk with UUID 1e44de19-7755-4219-9884-bc932958ac9e.
The disk is mounted at /mnt.

ERROR: /mnt/1e44de19-7755-4219-9884-bc932958ac9e                       is not a directory on 1e44de19-7755-4219-9884-bc932958ac9e.

Use Readline to obtain the line in an interactive shell

When you start your new backups:

Enter name of directory to store backups, relative to /run/media/<path> (to be created if not existing):

If you enter for example "baackup" and want to press the left arrow key it will add some "baackup^[[D". If you use read -e this error will be gone. Conside to use this on all interactive read operations.

does this only sync incremental snapshots?

thereby defeating the purpose? for example if my local drive gets hosed, what is the use-case of externally-backed incremental snapshots?

or is my understanding flawed?

info.xml file not copied

[alarm@alarmpi snap-sync]$ ls /mnt/backup/root/4/
snapshot

[alarm@alarmpi snap-sync]$ ls /.snapshots/4/
info.xml  snapshot

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.