45drives/cephgeorep

An efficient unidirectional remote backup daemon for CephFS.

cephgeorep's Issues

deletion support

If a file in the source is deleted, I don't see any support in cephgeorep to delete the file in the destination. Is that a planned future feature or am I missing something? Does a full rsync with --delete need to be run periodically for a full reconciliation with deletes?

Files and folders not in source dir are copied to destination.

Hello,

i am currently having the following issue:

Files and folders not in source dir are copied to destination.

here is a snipet of the config:

# local backup settings
Source Directory = /mnt/ceph           # path to the ceph directory you want backed up
Ignore Hidden = false         # ignore files beginning with "."
Ignore Windows Lock = true    # ignore files beginning with "~$"
Ignore Vim Swap = true        # ignore vim .swp files (.<filename>.swp)

# remote settings
Destination =  <removed because of ips>               # one or more backup targets (failover only)
# list of destinations can be space or comma separated and Destination can be
# defined multiple times to append more failover targets.
# Destination format: [[user@]host:][path]
# Destination = root@backup-gw1:/tank/backup,root@backup-gw2:/tank/backup

# daemon settings
Exec = rsync                  # program to use for syncing - rsync or scp
Flags = -a --relative -z        # execution flags for above program (space delim)
Metadata Directory = /var/lib/cephgeorep/ # put metadata on the ceph cluster if
                                          # you want to use pacemaker with
                                          # redundant gateways

It will start copying the files from the source and will then start to copy files from /root and then /.

So it will copy /mnt/ceph
then files in /root
and then everything in / this includes /root and /mnt/ceph.

This only happens when it is started via the service. When using cephfssyncd from the cli the issue does not occur.

Snapshot operation not permitted

Hi,

I'm trying to setup cephgeorep but I can't get it to work.

Output:

$ cephfssyncd -d -s -v

Reading last rctime from disk.
Watching: /mnt/cephroot
Checking for change.
Change detected in /mnt/cephroot
Creating snapshot: /mnt/cephroot/.snap/2928286snapshot1643381222.476912000
Error: Error creating snapshot path: Operation not permitted
Removing snapshot: /mnt/cephroot/.snap/2928286snapshot1643381222.476912000

What permissions are required for the ceph user? Or that is the issue? I've created a new user and allowed * on all objects but still didn't work.

More visibility on the status of backup (percentage finished/average speed, etc.)

I feel as though having some form of logging/verbosity on the status of Cephgeorep's file sync would be extremely helpful. Things like the list of files that it is currently working on a sync for, the transfer speed, any disconnects, etc. Currently user has no real way of knowing what is queued up for sync.

Rsync has some verbose outputs that could potentially be parsed for some of this such as the file lists, and if you use iotop you can potentially get the througput of just the georep daemon. If there was a central location you could go to get a nice overview of some of this information it would be great.