gilbertchen/benchmarking

A performance comparison of Duplicacy, restic, Attic, and duplicity

benchmarking's Issues

Oficial rules?

So I have been using this benchmark to optimize rdedup, and I was doing some test and improvements, etc. At some point, I'd like to add rdedup to the official "results". And I am wondering what are the actual rules? Eg. 1MB chunks are big, and rdedup results are much better with smaller chunks. I am guessing other tools might also benefit from different settings too.

As every benchmark encourages competition, it is only fair if the rules are clear.

For starter:

• can we tune settings for every program?
• do have to stick to the same settings in "linux" (small files) vs "vm" (big image)?

Comparison to duplicity and other similar named backup tool

Link: http://duplicity.nongnu.org/ and https://www.duplicati.com/
Recommendation: https://www.gigenet.com/blog/linux-encryption-backup-tools/

Add hashbackup

Altough not free when it will release, hashbackup has been a trustworthy tool for me.
Would be nice to include it in the backups.

www.hashbackup.com

Add Borg

I'm sure it's already on your todo list, but I'd just like to point out that most Attic users seem to have switched to Borg, which is an actively maintained fork, and one of the most recommended backup solutions in HN, Reddit and blogs.

Benchmark against rsync please...

Thanks

compare speed vs arqbakcup

Hello,

I am a user of arqbackup, and i am interested in a comparison with between the 2 as well, should that be possible.

Thank you in advance.

Record memory usage

Great comparisons, and as someone working with large backups and sometimes memory-constrained servers, details of peak memory usage would be v useful to see.

s/attic/borgbackup/

I found your benchmarks and noticed you refer to borgbackup 1.1.0b6 as "attic".

While attic is the project borg was forked from, it has developped way beyond attic meanwhile (and we're not finished yet). So maybe rather refer to it as borgbackup. While borg can import attic repos, the repo format is intentionally different (changed magic value).

Attic hasn't developped at all since the fork (still at 0.16) and has known/unfixed issues, so guess you do not need to include it in your benchmark any more.

Record original data size

In the comparison of how much data is used to store the backups, it would be nice to see some kind of baseline in how much the original data has grown.

Benchmarks for other non-similar sounding backup tools

For reference:
https://linuxhint.com/11_best_backup_tools_linux/
https://www.tecmint.com/linux-system-backup-tools/

results are empty

I'm trying to test my build but I think I do something wrong:

root@ubuntu-4gb-nbg1-1:~# git clone https://github.com/gilbertchen/benchmarking.git
Cloning into 'benchmarking'...
remote: Enumerating objects: 146, done.
remote: Total 146 (delta 0), reused 0 (delta 0), pack-reused 146
Receiving objects: 100% (146/146), 34.16 KiB | 4.27 MiB/s, done.
Resolving deltas: 100% (54/54), done.

root@ubuntu-4gb-nbg1-1:~/benchmarking# chmod +x linux-backup-test.sh

root@ubuntu-4gb-nbg1-1:~/benchmarking# mkdir test1

root@ubuntu-4gb-nbg1-1:~/benchmarking# ./linux-backup-test.sh test1
Cloning into '/root/benchmarking/test1/linux'...
remote: Enumerating objects: 12057, done.
remote: Counting objects: 100% (12057/12057), done.
remote: Compressing objects: 100% (4246/4246), done.
remote: Total 6683476 (delta 9340), reused 9230 (delta 7801), pack-reused 6671419
Receiving objects: 100% (6683476/6683476), 2.44 GiB | 20.28 MiB/s, done.
Resolving deltas: 100% (5523700/5523700), done.
Checking out files: 100% (64390/64390), done.
=========================================== init ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Checking out files: 100% (61071/61071), done.
Note: checking out '4f302921c1458d790ae21147f7043f4e6b6a1085'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 4f302921c145 Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus
======================================== backup 1 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Previous HEAD position was 4f302921c145 Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus
HEAD is now at 3481b6828523 xfs: move (and rename) the deferred bmap-free tracepoints
======================================== backup 2 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Checking out files: 100% (11519/11519), done.
Previous HEAD position was 3481b6828523 xfs: move (and rename) the deferred bmap-free tracepoints
HEAD is now at 46e36683f433 serial: earlycon: Extend earlycon command line option to support 64-bit addresses
======================================== backup 3 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Previous HEAD position was 46e36683f433 serial: earlycon: Extend earlycon command line option to support 64-bit addresses
HEAD is now at 566c56a493ea drm/i915: Sort DEV_INFO_FOR_EACH_FLAG
======================================== backup 4 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Checking out files: 100% (11262/11262), done.
Previous HEAD position was 566c56a493ea drm/i915: Sort DEV_INFO_FOR_EACH_FLAG
HEAD is now at 1be81ea58607 ARM: dts: imx6: Add imx-weim parameters to dtsi's
======================================== backup 5 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Previous HEAD position was 1be81ea58607 ARM: dts: imx6: Add imx-weim parameters to dtsi's
HEAD is now at ef3d232245ab mmc: mmc: Relax checking for switch errors after HS200 switch
======================================== backup 6 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Previous HEAD position was ef3d232245ab mmc: mmc: Relax checking for switch errors after HS200 switch
HEAD is now at 0e377f3b9ae9 iio: Add gravity sensor support
======================================== backup 7 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Checking out files: 100% (11816/11816), done.
Previous HEAD position was 0e377f3b9ae9 iio: Add gravity sensor support
HEAD is now at cb23ebdfa6a4 Drivers: staging: speakup: spk_priv.h - style fix
======================================== backup 8 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Checking out files: 100% (2585/2585), done.
Previous HEAD position was cb23ebdfa6a4 Drivers: staging: speakup: spk_priv.h - style fix
HEAD is now at 67db256ed1e0 devicetree/bindings: display: bridge: Add LVDS encoder DT bindings
======================================== backup 9 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Checking out files: 100% (11332/11332), done.
Previous HEAD position was 67db256ed1e0 devicetree/bindings: display: bridge: Add LVDS encoder DT bindings
HEAD is now at 1aed89640a89 mwifiex: apply radar flag
======================================== backup 10 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Previous HEAD position was 1aed89640a89 mwifiex: apply radar flag
HEAD is now at a6128f47f794 Merge branches 'thermal-core' and 'thermal-intel' into next
======================================== backup 11 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage
Checking out files: 100% (12067/12067), done.
Previous HEAD position was a6128f47f794 Merge branches 'thermal-core' and 'thermal-intel' into next
HEAD is now at 57caf4ec2b8b cpufreq: intel_pstate: Avoid division by 0 in min_perf_pct_min()
======================================== backup 12 ========================================
4.0K    /root/benchmarking/test1/linux-attic-storage
4.0K    /root/benchmarking/test1/linux-duplicacy-storage
4.0K    /root/benchmarking/test1/linux-duplicity-storage
4.0K    /root/benchmarking/test1/linux-rdedup-storage
4.0K    /root/benchmarking/test1/linux-restic-storage

 
root@ubuntu-4gb-nbg1-1:~/benchmarking# ./linux-backup-test.sh &> linux-backup-test.results

root@ubuntu-4gb-nbg1-1:~/benchmarking# cat linux-backup-test.results
Usage: ./linux-backup-test.sh <test dir>

root@ubuntu-4gb-nbg1-1:~/benchmarking# python tabulate.py linux-backup-test.results

root@ubuntu-4gb-nbg1-1:~/benchmarking# cat linux-backup-test.results
Usage: ./linux-backup-test.sh <test dir>