Giter Site home page Giter Site logo

bsync's Introduction

bsync

Bsync is a bidirectional file synchronization tool, using rsync for transfers. Moved files are also synchronized in a smart way.

It uses rsync for file transfers, GNU find to generate filelist snapshots, and ssh for remote transfers.

bsync is an alternative to Unison, written in Python 3. A big strength of bsync: it can detect and apply moved files from one side to the other (Unison uses some copy calls to handle moved files).

I developped it to be able to synchronize my music directory from my laptop to my Raspberry Pi in an efficient way, and to sync with my girlfriend laptop too.

Bsync is released under GPL. Feel free to report any bugs/wishes in GitHub issues.

Install

wget https://raw.github.com/dooblem/bsync/master/bsync
chmod +x bsync

For remote syncing: don't forget to install rsync.

Usage

Fairly simple:

./bsync DIRECTORY1 DIRECTORY2
./bsync ALICE_DIR  bob@sshserver:BOB_DIR

bsync can also be used to sync with a master directory:

# Alice makes local changes
./bsync ALICE_DIR MASTER_DIR
./bsync BOB_DIR   MASTER_DIR
# Bob gets Alice changes, sending his changes to master in the same time

bsync help and options:

Usage: bsync [options] DIR1 DIR2

	DIR can be user@sshserver:DIR
	-v                Verbose
	-i                Ignore permissions
	-p PORT           Port for SSH
	-o SSHARGS        Custom options for SSH
	--rsync-args=ARGS WARNING some rsync arguments may break syncing

Features

  • Moved files detection (using inodes numbers)
  • Remote directories using SSH
  • No problem with symlinks or permissions
  • Conflict detection
  • Python not needed on remote side (just GNU find and rsync)
  • Exclude some subdirectories from sync (just create a .bsync-ignore file)
  • Move your sync dirs without loosing sync memory (filelists stored inside directories in .bsync-snap-* files)
  • Auto disable permissions on fat filesystems
  • MacOSX support (requirement: GNU find installed)

Limitations:

  • files ownership ignored (would matter if syncing from root user, but sufficient for regular users)
  • no subdir conflict detection (a bit like in git where only files matter, no conflict is detected if dir1/dir/ removed and dir2/dir/file created the other side)
  • No Windows support
  • Not tested under: OpenBSD, FreeBSD (any feedback appreciated)

Example

$ ./bsync dir1 dir2
Loading filelists...
dir1                        dir2                   
new                    -->                         (copy)
subdir/a               -->  subdir/a               (sync)
                       <--  newdir/                (mkdir)
                       <--  newdir/newfile         (copy)
Apply actions? [y/N] y
Applying actions...
rsync: new
rsync: subdir/a
rsync: newdir/newfile
Updating filelists...
Done!

$ ./bsync dir1 dir2
Loading filelists...
Identical directories. Nothing to do.

Conflict handling

Bsync prompts the user for conflicts.

A sample run with a conflict: file deleted one side and updated the other side.

$ bsync dir1/ dir2/
Loading filelists...

Conflicting changes on: testfile
*deleted*                           <?>   -rw-r--r-- 7B (2014-01-30 18:47:40) (conflict)
Which one do I keep [1/2/?] ?
	1	Keep left version
	2	Keep right version
	1a	Keep left version for all
	2a	Keep right version for all
Which one do I keep [1/2/?] 2
dir1/                                    dir2/                               
                                    <--  testfile                            (copy)
Apply actions? [y/N] y
Applying actions...
rsync: testfile
Updating filelists...
Done!

.bsync-ignore files

You can add directories/files paths in a .bsync-ignore file located at the root of a sync directory. Every path in it will be ignored when syncing with other dirs. You can also see that as a mask for the synchronization.

Say, if I have a dir1/.bsync-ignore file with content:

path/to/ignoredir
path/to/ignorefile

dir1/path/to/ignoredir (+content) and dir1/path/to/ignorefile will be ignored in the next bsync runs.

The ignore file has to be very simple. No comments, just path prefixes.

See also

My blog

bsync's People

Contributors

coreagile avatar dooblem avatar jashort avatar msierks avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bsync's Issues

bsync daemon

it will be nice to have bsync as a daemon and then using rest api to fix conflits and use FS notify to auto detect changes and sync files almost after file is just closed.

file move into subfolder doesn't seem to be detected

Hi

I've moved some videos to a subfolder, and bsync wants to duplicate these videos.
Is it normal that bsync proposes to copy files with same name and size in both directions (in different folders)?
In these case, an optional additionnal test would be to checksum these files and if they are identical, do a file move.

[enhancement] Avoid remote deletion

Rsync doesn't delete removed file by default in the second destination, so why bsync can't do it?
When you remove unused files on your laptop that could be fine if there is always a copy on the server.

desired behavior:
If a flag is set (ex: -k for "keep"), removed files are wrote in .bsync-ignore & withdrawn from .bsync-snap-*.
You can retrieve later local removed files by deleting them from .bsync-ignore.

Some moved/renamed/deleted files are not detected

Using last Bsync on two similar Ubuntu 16.04.5 LTS, underlying filesystem is Btrfs.

$ bsync /dir -i -v -b user@hostname:/dir
Loading filelists...
Loading original filelist from snap files...
Loading .bsync-snap-20210708181831.750984...
Loading dir1 filelist...
Loading dir2 filelist...
Analysing original paths...
Analysing remaining new paths in dir1...
Analysing remaining new paths in dir2...

(LEFT DIR CONTENT)                                                                                                                                         (RIGHT DIR CONTENT)                                                                                                                                   (ACTION)

mailbox/username/Maildir/cur/1625768344.H43283P9597.xxxxxx:2,Tbcj                                                       -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768373.M569836P4621.xxxxxx:2,aci                         -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768373.M569837P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768373.M569838P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768373.M569839P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768373.M569840P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768373.M569841P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768375.H897729P9502.xxxxxx:2,Tbcj                                                      -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625765442.H606313P9101.xxxxxx:2,                                                          <--                                                                                                                                                        (rm)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765172.H923861P9051.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765428.H151405P9093.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765442.H606313P9101.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765772.H980977P9169.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765965.H849086P9202.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625766466.H954726P9290.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625767067.H976495P9394.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625767849.H415399P9524.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625768344.H43283P9597.xxxxxx                                                              (copy)

Todo in /dir: rm:1 cp:9
Todo in user@hostname:/dir: cp:8

Applying actions...
Applying actions in dir2...
rsync: mailbox/username/Maildir/cur/1625768344.H43283P9597.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768373.M569836P4621.xxxxxx:2,aci
rsync: mailbox/username/Maildir/cur/1625768373.M569837P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768373.M569838P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768373.M569839P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768373.M569840P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768373.M569841P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768375.H897729P9502.xxxxxx:2,Tbcj
Applying actions in dir1...
rsync: mailbox/username/Maildir/new/1625765172.H923861P9051.xxxxxx
rsync: mailbox/username/Maildir/new/1625765428.H151405P9093.xxxxxx
rsync: mailbox/username/Maildir/new/1625765442.H606313P9101.xxxxxx
rsync: mailbox/username/Maildir/new/1625765772.H980977P9169.xxxxxx
rsync: mailbox/username/Maildir/new/1625765965.H849086P9202.xxxxxx
rsync: mailbox/username/Maildir/new/1625766466.H954726P9290.xxxxxx
rsync: mailbox/username/Maildir/new/1625767067.H976495P9394.xxxxxx
rsync: mailbox/username/Maildir/new/1625767849.H415399P9524.xxxxxx
rsync: mailbox/username/Maildir/new/1625768344.H43283P9597.xxxxxx
Updating filelists...
Updating snap files: .bsync-snap-20210708182009.368791...
Done!
Cleaning SSH master...

Few seconds after:

$ bsync /dir -i -v -b user@hostname:/dir
Loading filelists...
Loading original filelist from snap files...
Loading .bsync-snap-20210708182009.368791...
Loading dir1 filelist...
Loading dir2 filelist...
Analysing original paths...
Analysing remaining new paths in dir1...
Analysing remaining new paths in dir2...

(LEFT DIR CONTENT)                                                                                                                                         (RIGHT DIR CONTENT)                                                                                                                                   (ACTION)

mailbox/username/Maildir/cur/1625768409.M755002P4621.xxxxxx:2,aci                         -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768409.M755003P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768409.M755004P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768409.M755005P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768409.M755006P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768409.M755007P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625768409.M755008P4621.xxxxxx:2,Tbcj                        -->                                                                                                                                                        (copy)
mailbox/username/Maildir/cur/1625765442.H606313P9101.xxxxxx:2,                                                          <--                                                                                                                                                        (rm)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765172.H923861P9051.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765428.H151405P9093.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765442.H606313P9101.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765772.H980977P9169.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625765965.H849086P9202.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625766466.H954726P9290.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625767067.H976495P9394.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625767849.H415399P9524.xxxxxx                                                             (copy)
                                                                                                                                                      <--  mailbox/username/Maildir/new/1625768344.H43283P9597.xxxxxx                                                              (copy)

Todo in /dir: rm:1 cp:9
Todo in user@hostname:/dir: cp:7

Applying actions...
Applying actions in dir2...
rsync: mailbox/username/Maildir/cur/1625768409.M755002P4621.xxxxxx:2,aci
rsync: mailbox/username/Maildir/cur/1625768409.M755003P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768409.M755004P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768409.M755005P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768409.M755006P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768409.M755007P4621.xxxxxx:2,Tbcj
rsync: mailbox/username/Maildir/cur/1625768409.M755008P4621.xxxxxx:2,Tbcj
Applying actions in dir1...
rsync: mailbox/username/Maildir/new/1625765172.H923861P9051.xxxxxx
rsync: mailbox/username/Maildir/new/1625765428.H151405P9093.xxxxxx
rsync: mailbox/username/Maildir/new/1625765442.H606313P9101.xxxxxx
rsync: mailbox/username/Maildir/new/1625765772.H980977P9169.xxxxxx
rsync: mailbox/username/Maildir/new/1625765965.H849086P9202.xxxxxx
rsync: mailbox/username/Maildir/new/1625766466.H954726P9290.xxxxxx
rsync: mailbox/username/Maildir/new/1625767067.H976495P9394.xxxxxx
rsync: mailbox/username/Maildir/new/1625767849.H415399P9524.xxxxxx
rsync: mailbox/username/Maildir/new/1625768344.H43283P9597.xxxxxx
Updating filelists...
Updating snap files: .bsync-snap-20210708182100.993313...
Done!
Cleaning SSH master...

1625765442.H606313P9101.xxxxxx was remove or move from dir1 between each sync.

Any idea of what's going on?

Keeping track of mount information

As I remember from Unison, mount status of the directories (recursively) must be checked before (or even during?) synchronizing. If there is something different, I must be warned seriously (for example force user to type: "yes i know") if there is a different or missing mount entry either in localhost or remote host.

Because this problem leads unwanted erase of subfolders.

Example:
We have DIR1 and DIR2 to sync.
DIR1/aaa/bbb is mounted via sshfs from a remote machine, or with "-o bind" option from localhost.
Sync the two directories.
Umount DIR1/aaa/bbb (it is unmounted because a network/cable connection error)
Sync again: Problem.
We should be warned about that.

What is the format of the .bsync-ignore file?

I'm trying to remove folders called node_modules and .npm from my backup. However bsync always wants to copy them across?!

Especially keen to know please, can I use:

# for comments?
$ and ^ for start and end of line like in regex?
? for a single char random
* for any char random in this path?
/ naturally this delimits the directories. or does it?

Here is my .bsync-ignore file, which seems to half work:

nginx/fastcgi.conf
nginx/fastcgi_params
nginx/koi-utf
nginx/koi-win
nginx/mime.types
# nginx/nginx.conf <----- this is the file we want!
# nginx/proxy_params <----- this is the file we want!
nginx/scgi_params
nginx/uwsgi_params
nginx/win-utf


proc
.meteor/*
snort
.npm/*
node_modules/*
/home/cc/.*
/home/cc/crowdcourier/.*
/home/cc/crowdcourier/app/.*
crowdcourier/.*

# ROOT OF FILE SYSTEM ################

# root
bin
boot
dev
# etc
home
initrd.img
initrd.img
lib
lib64
lost+found
media
mnt
opt
proc
root
run
sbin
snap
srv
sys
tmp
usr
var
vmlinuz
vmlinuz.ol
webmin-set


# CHEESE FILES
anacrontab
appstream.conf
bluetooth
anacrontab
appstream.conf
bluetooth
brlapi.key
brltty
brltty.conf
catdocrc
chatscripts
cracklib
cryptsetup-initramfs
cups
cupshelpers
$dconf
doc-base
drbl
fstab-backup
fstab-hot
fstab-main-system
fwupd
fwupd.conf
gconf
gdb
gdm3
geoclue
ghostscript
grub-customizer
hddtemp.db
hp
ifplugd
ImageMagick-6
java-8-openjdk
kde4rc
kerneloops.conf
ksysguarddrc
kubuntu-default-settings
libao.conf
libpaper.d
libreoffice
lintianrc
menu-methods
mpd.conf
mtools.conf
netplan
netscsid.conf
nilfs_cleanerd.conf
PackageKit
papersize
pcmcia
pnm2ppa.conf
python3.6
sddm
signon-ui
speech-dispatcher
sudoers
sudoers.dist
sysstat
thunderbird
timidity
usb_modeswitch.conf
usb_modeswitch.d
whoopsie
wildmidi
wodim.conf
wpa_supplicant
xfe


# DOTFILES
.java/
.swp

# files
popularity-contest.conf
mailcap.order
group
legal
ld.so.conf
nsswitch.conf
wgetrc
shadow-
mke2fs.conf
os-release
lsb-release
gshadow-
apg.conf
updatedb.conf
magic
ca-certificates.conf.dpkg-old
insserv.conf
mime.types
securetty
vtrgb
bmon.conf
locale.gen
bindresvport.blacklist
hdparm.conf
subgid-
mailcap
subuid
shadow
networks
iptables.up.rules
ucf.conf
subgid
rpc
qemu-ifup
modules
cczerc
ld.so.cache
login.defs
crypttab
passwd-
drirc
mtab
.bsync-snap-*
subuid-
passwd
gshadow
rmt
inputrc
pam.conf
.pwd.lock
protocols
localtime
resolv.conf
ethertypes
overlayroot.conf
at.deny
locale.alias
magic.mime
machine-id
vdpau_wrapper.cfg
services
ltrace.conf
group-
manpath.config
libaudit.conf
ip6tables.up.rules
qemu-ifdown
rc.local
list.txt
request-key.conf
timezone
gai.conf
popularity-contest.conf
adduser.conf
mailcap.order
kernel-img.conf
group
screenrc
crontab
profile
legal
ld.so.conf
debian_version
nsswitch.conf
wgetrc
issue.net
sos.conf
shadow-
mke2fs.conf
os-release
hostname
lsb-release
gshadow-
apg.conf
updatedb.conf
debconf.conf
magic
ca-certificates.conf.dpkg-old
insserv.conf
deluser.conf
mime.types
environment
securetty
fstab
vtrgb
issue
bmon.conf
locale.gen
bindresvport.blacklist
hdparm.conf
subgid-
mailcap
bash_completion
subuid
shadow
networks
iptables.up.rules
ucf.conf
subgid
rpc
qemu-ifup
modules
sysctl.conf
cczerc
ld.so.cache
login.defs
logrotate.conf
crypttab
passwd-
drirc
mtab
subuid-
passwd
gshadow
rmt
inputrc
bash.bashrc
pam.conf
rsyslog.conf
.pwd.lock
protocols
sensors3.conf
ca-certificates.conf
localtime
zsh_command_not_found
.bsync-snap-20180410221645.364918
resolv.conf
ethertypes
overlayroot.conf
at.deny
locale.alias
magic.mime
machine-id
vdpau_wrapper.cfg
files.txt
services
ltrace.conf
group-
manpath.config
libaudit.conf
nanorc
ip6tables.up.rules
fuse.conf
host.conf
qemu-ifdown
rc.local
request-key.conf
timezone
gai.conf


# DIRECTORIES

at-spi2
network
terminfo
init.d
vim
iscsi
rc5.d
ppp
dnsmasq.d
firefox
request-key.d
xml
polkit-1
rc1.d
mysql
sane.d
init
systemd
apport
xdg
fonts
gtk-3.0
sysctl.d
lvm
pm
insserv.conf.d
kernel
ufw
dictionaries-common
modules-load.d
samba
cron.monthly
rcS.d
esound
skel
UPower
sensors.d
gss
ldap
console-setup
calendar
gtk-2.0
byobu
logrotate.d
python3.5
binfmt.d
libnl-3
apparmor.d
dpkg
ca-certificates
vmware-tools
sasl2
apache2
cron.weekly
cron.hourly
depmod.d
udisks2
pulse
texmf
groff
zsh
tmpfiles.d
python
python2.7
rc3.d
rc2.d
insserv
rsyslog.d
ssl
update-motd.d
selinux
rc0.d
acpi
cron.d
lighttpd
resolvconf
libvirt
apparmor
pam.d
initramfs-tools
pki
X11
apt
python3
default
docker
sgml
NetworkManager
dhcp
grub.d
emacs
openal
ssh
webmin
apm
bash_completion.d
modprobe.d
mdadm
dkms
sudoers.d
update-manager
kbd
logcheck
cron.daily
lynx
security
rc4.d
opt
mplayer
udev
avahi
rc6.d
profile.d
newt
dbus-1
iproute2
ld.so.conf.d
gnome
update-notifier
alternatives
dnsmasq.d-available
perl

On macOS, how do I know if I have GNU Find installed?

If I type find I get a find. It looks like I have one installed via Brew.

/u/l/bin ❯❯❯ ls -la find
lrwxr-xr-x  1 tom  access_bpf  34  2 Sep  2016 find -> ../Cellar/findutils/4.6.0/bin/find

I ask before when I try to run bsync I get error:

zsh: /usr/local/bin/bsync: bad interpreter: /usr/bin/python3: no such file or directory

/u/l/bin ❯❯❯ brew install python3                                                                                                                                                                                                                                              ✘ 127
Error: python 2.7.14_3 is already installed
To upgrade to 3.6.5, run `brew upgrade python`

/u/l/bin ❯❯❯ bsync
zsh: /usr/local/bin/bsync: bad interpreter: /usr/bin/python3: no such file or directory
[[email protected]:/usr/local/bin]
/u/l/bin ❯❯❯ python                                                                                                                                                                                                                                                            ✘ 127
Python 2.7.10 (default, Feb  7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
KeyboardInterrupt
>>> exit
Use exit() or Ctrl-D (i.e. EOF) to exit
>>> ^D
[[email protected]:/usr/local/bin]
/u/l/bin ❯❯❯ python3
Python 3.6.5 (default, Mar 30 2018, 06:42:10)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

bsync over ssh doesn't use existing 'ControlPath'

E.g. I have a ~/.ssh/config set up so:

ControlMaster auto
ControlPath /tmp/ssh_mux_%h_%p_%r
ControlPersist 1h
ForwardAgent yes

Host rs
Hostname remoteserverip
User myusername

It does use the host shortcut I set up, but it doesn't respect/utilize the control persist I am using. This should use the existing connection instead of creating a new one and requesting my password every time.

Support for hooking an external script

Hi.

Considered that is difficult to add support for ACL synchronization to bsync, I propose to add support for running an external script between apply_small_actions and apply_rsync_actions.
The external script could take care of ACL synchronization, and if I'm not wrong the best moment to do it is between the fast moves and deletion (apply_small_actions) and the possible slow rsync operations, to reduce the time window where ACL are not synchronized.
If using --acls rsync argument, is not needed to resync ACL after rsync operations.

The external script could be specified by a command line option -s, for example.

If you agree, I can branch and propose a pull request.
(I did now another pull request for batch mode and autoanswer, as discussed in another thread)

Stefano

Feedback on the use of special caracters in files' name (HFS+ <-> ext4)

Feedback on the use of bsync between a mac with HFS+ filesystem and a NAS with ext4 partition.

When using special caracters as ñ, ó or í, but not á (forinstance), the synchrnization becomes very unstable through ssh. Move change, rmdir and so on, give errors.

On the contrary, between my laptop with Debian and ext4 partition there is no problem.

Thus, I guess there's a problem with caracter encoding between both type of partition (HFS and ext4) and bsync does not manage it.

Symbolic link on a ntfs external hard drive

Hi,
I am a new user of bsync and I find it extremely useful.
I don't have any issue when synchronizing two machines running under ubuntu 16.04.
Now I am trying to synchronize with my external USB hard drive, ntfs formatted.

I have the following issue :

  • at the first synchronization, all seem to go well
  • at the second, I get the following error message :

Error: difference in snaps for path:

followed by the name of a file that is a symbolic link. And no way to get rid of it.

I've made some tests : this message does not appear when synchronizing to a USB key formatted ext4, but appear with the ones formatted ntfs.

I really would like to keep my external hard drive ntfs because I need to make regularly transfer with machines running Windows OS.
How do you understand the error message I get ? Is there a way to pass by ?

Thanks. (below, the complete session)

cleobuline 21/11/2016 17:08:44 /home/guy : bsync -iv 01-Math/ /media/guy/Hypathie/01-Math/
Loading filelists...
Loading original filelist from snap files...
Old filelist not found. Starting with empty history.
Loading dir1 filelist...
Loading dir2 filelist...
Analysing original paths...
Analysing remaining new paths in dir1...
Analysing remaining new paths in dir2...

(LEFT DIR CONTENT) (RIGHT DIR CONTENT) (ACTION)

programme_agregation.txt~ --> (copy)
liens_et_docs_20161017.txt~ --> (copy)

Todo in 01-Math/:
Todo in /media/guy/Hypathie/01-Math/: cp:2
Apply actions? [y/N] y

Applying actions...
Applying actions in dir2...
rsync: liens_et_docs_20161017.txt~
rsync: programme_agregation.txt~
Applying actions in dir1...
Updating filelists...
Updating snap files: .bsync-snap-20161121170918.712810...
Done!

First is OK... Now I synchronize a second time :

cleobuline 21/11/2016 17:09:20 /home/guy :
cleobuline 21/11/2016 17:09:20 /home/guy : bsync -iv 01-Math/ /media/guy/Hypathie/01-Math/
Loading filelists...
Loading original filelist from snap files...
Loading .bsync-snap-20161121170918.712810...
Error: difference in snaps for path: Geometrie/Combes_Algebre_et_geometrie_ed1_v1.pdf
cleobuline 21/11/2016 17:09:23 /home/guy :
cleobuline 21/11/2016 17:09:24 /home/guy :

This file Geometrie/Combes_Algebre_et_geometrie_ed1_v1.pdf is a symbolic link

Find error (permission issue?) makes bsync terminate

I'll dig when I have time:

Loading filelists...
Old filelist not found. Starting with empty history.
find: `ceremcem/httpd/ceremcem.net/ağkökü/phpmathpublisher/doc': Permission denied
find: `ceremcem/httpd/ceremcem.net/ağkökü/phpmathpublisher/doc_fr': Permission denied
Find Error in ceremcem/

Permission may be denied, but sync must keep going on as much as possible.

Possible to have filename.* at .bsync-ignore

Dear Sirs,
is it possible at .bsync-ignore file to have wildcard character in the filename e.x.

dovecot.*

in order to get ignored all dovecot.index , dovecot.index.log etc files?

Thanks

Add support for FTP-SSL (AUTH TLS)

Hi,

I'd like to use bsync to sync files on my hoster, but my hoster only allows me to transfer files using FTPS, not SFTP.

Unfortunately, on the attempt to connect I get this error message, so I assume FTPS is not supported:

$ ./bsync www ftps://username:[email protected]
ssh: Could not resolve hostname ftps: nodename nor servname provided, or not known
Error: could not open SSH connection.

I'd love to see this being implemented, or at least, get a suitable alternative that is a binary that requires no installation whatsoever.

Permissions and ownership not correctly synched

Hi

When changing only permission and ownership in one side, bsync doesn't detect the modification and doesn't propagate it.

I'm doing it with a separate bash script: it synchronizes only files differents by permissions or ownership, and uses rsync and some trick to force synchronization in the right direction (I can't let rsync decide the direction of synchronization based on modification time, because obviously we need to check the change time).

Here is the bash script I'm using: probably not possible to use any of it to integrate in bsync, but I show just as an example. Also, this is a simple, not updated version, that only synchronize in one direction from remote to local.

#!/bin/bash
scriptname="$(basename "$(test -L "$0" && readlink "$0" || echo "$0")")"
local_path="$1"
remote_ip="$2"
remote_path="$3"
RSYNC_ARGS="$4"

[ -z "$RSYNC_ARGS" ] && RSYNC_ARGS="-a -A -X -z -E"

local_path="${local_path%%/}"
remote_path="${remote_path%%/}"
tmp_file="/tmp/${scriptname}_$RANDOM"

/usr/bin/rsync --rsync-path="rsync" $RSYNC_ARGS -ui -e "/usr/bin/ssh -C -i /root/.ssh/id_rsa -p 22" \
    "$local_path/" root@$remote_ip:"$remote_path/" --dry-run | \
    grep -Ev "^[^ ]*(c|s|t)[^ ]* " | \
    grep -E "^[^ ]*(p|o|g|a)[^ ]* " | \
    sed -e 's/^[^ ]* //' >"$tmp_file"

cd "$local_path"
cat "$tmp_file" | xargs -I {} stat -c '%n|%Z|%Y' "{}" | sort >"${tmp_file}.1"

cat "$tmp_file" | /usr/bin/ssh -C -i /root/.ssh/id_rsa -p 22 root@$remote_ip "cd \"$remote_path\" ; xargs -I {} stat -c '%n|%Z|%Y' \"{}\"" | sort >"${tmp_file}.2"

join -j 1 -t \| -o 1.1,1.2,2.2 "${tmp_file}.1" "${tmp_file}.2" | awk -F'|' '{if ($2 < $3) print $1}' | \
    /usr/bin/rsync --rsync-path="rsync" $RSYNC_ARGS -ui -e "/usr/bin/ssh -C -i /root/.ssh/id_rsa -p 22" \
    root@$remote_ip:"$remote_path/" "$local_path/" --files-from=-

rm -f "$tmp_file"*

bsync errors out on conflict

The following sequence crashes bsync:

#! /bin/sh

rm -rf testdir1 testdir2
mkdir testdir1 testdir2
echo xxxxx > testdir1/data
bsync testdir1 testdir2
sleep 2
rm -rf testdir1
touch testdir2/data
bsync testdir1 testdir2

Interrupting synchronization causes permanent tmp files

If we interrupt synchronization by using Ctrl+C, temporary files that are used by rsync are left in the filesystem. Once we restart sync process, this tmp file is propagated between folders.

Maybe bsync should use a temporary folder to copy the file, and then use mv. That will be atomic.

Keep a watch

I've used bsync and it is awesome. How can we use it to keep a watch on the local changes and synchronize the host and remote on real-time?

[Wishlist] Sync summary

From Cerem Cem ASLAN idea :

  1. When about to sync directories, change list should be redisplayed briefly. If I expect “there should be no copy from DIR1 to DIR2″, I should see it in the last part, before “yes/no” prompt, something like:

    file copies –> : 3
    file copies <– : 89
    yes/no?

ACL and batch mode: improvement proposal

bsync.txt

Hi

I propose following changes, and attach a diff file bsync.txt with a suggested implementation:

  1. -b (batch mode) should in my opinion skip the "Apply actions" prompt (or alternatively would be better provide a silent option)
  2. I added --acls, --xattrs and --compress to rsync arguments, just because I needed them: would be nice to add them by default or allow them optionally
  3. in conflict prompt I added option "r" and "ra" to allow automatic resolution of conflicts based on following rules:
  • in case of deletion on one side, and update on another side, choose sync of updated file instead of deletion of removed file
  • in case of updated file in both sides, privilege the most recent file
  • in other cases, give precedence to left side
  1. I added a -a (autoanswer) option, with which is possible to decide the answer to all conflicts prompts (1, 2 or r)

I know, it can be very dangerous to allow automatic resolution of conflicts, but that's what I need and I think is very helpful to give possibility to a completely background run.

A feature to backup conflicted files and directory would be wonderful. In rsync there is --backup option, but I think is not useful in the cases when rsync is called inside bsync.

I never coded in python, so be indulgent with provided code.

PS
bsync is one of few programs I could find that allow remote and bidirectional synchronization based on rsync with move and rename detection of files and directory.

Thank you for sharing it

too long for Unix domain socket

I get the following error when I do
./bsync /sdcard/work/zot/ home:/home/abhishek/work/zot

unix_listener: "/data/data/com.termux/files/usr/tmp/tmpu4bc4w1x/bsync_abhishek@mybighostname:myport.0texbz3aTGia5AqP" too long for Unix domain socket
Error: could not open SSH connection.

bsync suport 3 nodes

Hi it will be nice to support multiple nodes sync
left center right when solving conflicts.

is there a way to ignore hidden files

when i use the script for my home folder on linux , it copies all .file (hidden files) ,if you add this option to ignore hidden files , i would be very happy.

Ignore by file extension?

I've tried

*.lock and *.sr.lock and .lock and .sr.lock

but it still tries to sync them, any help?

bsync fails on top-level paths with spaces

Using latest master as of 2021/01/01 on Ubuntu 20.04 (both sides).
rsync version 3.1.3 protocol version 31

If remote directory has a space, transfer fails as shown below. If I rename the remote dir to remove the space, transfer succeeds. I've tried with quotes and with an escaped (backslash) space on the command-line.

$ bsync -v Calibre/ 'remotehost:Calibre Library/'
user@remotehost's password: 
Loading filelists...
Loading original filelist from snap files...
Old filelist not found. Starting with empty history.
Loading dir1 filelist...
Loading dir2 filelist...

.... skipped some conflict-resolution questions and list of pending transfers ...

Todo in Calibre/: sync:5
Todo in remotehost:Calibre Library/: cp:260 sync:2
Apply actions? [y/N] y

Applying actions...
Applying actions in dir2...
Applying actions in dir1...
rsync  version 3.1.3  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.

Usage: rsync [OPTION]... SRC [SRC]... DEST
  or   rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
  or   rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
  or   rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
  or   rsync [OPTION]... [USER@]HOST:SRC [DEST]
  or   rsync [OPTION]... [USER@]HOST::SRC [DEST]
  or   rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect
to an rsync daemon, and require SRC or DEST to start with a module name.

Options
 -v, --verbose               increase verbosity
     --info=FLAGS            fine-grained informational verbosity
     --debug=FLAGS           fine-grained debug verbosity
     --msgs2stderr           special output handling for debugging
 -q, --quiet                 suppress non-error messages
     --no-motd               suppress daemon-mode MOTD (see manpage caveat)
 -c, --checksum              skip based on checksum, not mod-time & size
 -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
     --no-OPTION             turn off an implied OPTION (e.g. --no-D)
 -r, --recursive             recurse into directories
 -R, --relative              use relative path names
     --no-implied-dirs       don't send implied dirs with --relative
 -b, --backup                make backups (see --suffix & --backup-dir)
     --backup-dir=DIR        make backups into hierarchy based in DIR
     --suffix=SUFFIX         set backup suffix (default ~ w/o --backup-dir)
 -u, --update                skip files that are newer on the receiver
     --inplace               update destination files in-place (SEE MAN PAGE)
     --append                append data onto shorter files
     --append-verify         like --append, but with old data in file checksum
 -d, --dirs                  transfer directories without recursing
 -l, --links                 copy symlinks as symlinks
 -L, --copy-links            transform symlink into referent file/dir
     --copy-unsafe-links     only "unsafe" symlinks are transformed
     --safe-links            ignore symlinks that point outside the source tree
     --munge-links           munge symlinks to make them safer (but unusable)
 -k, --copy-dirlinks         transform symlink to a dir into referent dir
 -K, --keep-dirlinks         treat symlinked dir on receiver as dir
 -H, --hard-links            preserve hard links
 -p, --perms                 preserve permissions
 -E, --executability         preserve the file's executability
     --chmod=CHMOD           affect file and/or directory permissions
 -A, --acls                  preserve ACLs (implies --perms)
 -X, --xattrs                preserve extended attributes
 -o, --owner                 preserve owner (super-user only)
 -g, --group                 preserve group
     --devices               preserve device files (super-user only)
     --copy-devices          copy device contents as regular file
     --specials              preserve special files
 -D                          same as --devices --specials
 -t, --times                 preserve modification times
 -O, --omit-dir-times        omit directories from --times
 -J, --omit-link-times       omit symlinks from --times
     --super                 receiver attempts super-user activities
     --fake-super            store/recover privileged attrs using xattrs
 -S, --sparse                turn sequences of nulls into sparse blocks
     --preallocate           allocate dest files before writing them
 -n, --dry-run               perform a trial run with no changes made
 -W, --whole-file            copy files whole (without delta-xfer algorithm)
     --checksum-choice=STR   choose the checksum algorithms
 -x, --one-file-system       don't cross filesystem boundaries
 -B, --block-size=SIZE       force a fixed checksum block-size
 -e, --rsh=COMMAND           specify the remote shell to use
     --rsync-path=PROGRAM    specify the rsync to run on the remote machine
     --existing              skip creating new files on receiver
     --ignore-existing       skip updating files that already exist on receiver
     --remove-source-files   sender removes synchronized files (non-dirs)
     --del                   an alias for --delete-during
     --delete                delete extraneous files from destination dirs
     --delete-before         receiver deletes before transfer, not during
     --delete-during         receiver deletes during the transfer
     --delete-delay          find deletions during, delete after
     --delete-after          receiver deletes after transfer, not during
     --delete-excluded       also delete excluded files from destination dirs
     --ignore-missing-args   ignore missing source args without error
     --delete-missing-args   delete missing source args from destination
     --ignore-errors         delete even if there are I/O errors
     --force                 force deletion of directories even if not empty
     --max-delete=NUM        don't delete more than NUM files
     --max-size=SIZE         don't transfer any file larger than SIZE
     --min-size=SIZE         don't transfer any file smaller than SIZE
     --partial               keep partially transferred files
     --partial-dir=DIR       put a partially transferred file into DIR
     --delay-updates         put all updated files into place at transfer's end
 -m, --prune-empty-dirs      prune empty directory chains from the file-list
     --numeric-ids           don't map uid/gid values by user/group name
     --usermap=STRING        custom username mapping
     --groupmap=STRING       custom groupname mapping
     --chown=USER:GROUP      simple username/groupname mapping
     --timeout=SECONDS       set I/O timeout in seconds
     --contimeout=SECONDS    set daemon connection timeout in seconds
 -I, --ignore-times          don't skip files that match in size and mod-time
 -M, --remote-option=OPTION  send OPTION to the remote side only
     --size-only             skip files that match in size
 -@, --modify-window=NUM     set the accuracy for mod-time comparisons
 -T, --temp-dir=DIR          create temporary files in directory DIR
 -y, --fuzzy                 find similar file for basis if no dest file
     --compare-dest=DIR      also compare destination files relative to DIR
     --copy-dest=DIR         ... and include copies of unchanged files
     --link-dest=DIR         hardlink to files in DIR when unchanged
 -z, --compress              compress file data during the transfer
     --compress-level=NUM    explicitly set compression level
     --skip-compress=LIST    skip compressing files with a suffix in LIST
 -C, --cvs-exclude           auto-ignore files the same way CVS does
 -f, --filter=RULE           add a file-filtering RULE
 -F                          same as --filter='dir-merge /.rsync-filter'
                             repeated: --filter='- .rsync-filter'
     --exclude=PATTERN       exclude files matching PATTERN
     --exclude-from=FILE     read exclude patterns from FILE
     --include=PATTERN       don't exclude files matching PATTERN
     --include-from=FILE     read include patterns from FILE
     --files-from=FILE       read list of source-file names from FILE
 -0, --from0                 all *-from/filter files are delimited by 0s
 -s, --protect-args          no space-splitting; only wildcard special-chars
     --address=ADDRESS       bind address for outgoing socket to daemon
     --port=PORT             specify double-colon alternate port number
     --sockopts=OPTIONS      specify custom TCP options
     --blocking-io           use blocking I/O for the remote shell
     --stats                 give some file-transfer stats
 -8, --8-bit-output          leave high-bit chars unescaped in output
 -h, --human-readable        output numbers in a human-readable format
     --progress              show progress during transfer
 -P                          same as --partial --progress
 -i, --itemize-changes       output a change-summary for all updates
     --out-format=FORMAT     output updates using the specified FORMAT
     --log-file=FILE         log what we're doing to the specified FILE
     --log-file-format=FMT   log updates using the specified FMT
     --password-file=FILE    read daemon-access password from FILE
     --list-only             list the files instead of copying them
     --bwlimit=RATE          limit socket I/O bandwidth
     --stop-at=y-m-dTh:m     Stop rsync at year-month-dayThour:minute
     --time-limit=MINS       Stop rsync after MINS minutes have elapsed
     --outbuf=N|L|B          set output buffering to None, Line, or Block
     --write-batch=FILE      write a batched update to FILE
     --only-write-batch=FILE like --write-batch but w/o updating destination
     --read-batch=FILE       read a batched update from FILE
     --protocol=NUM          force an older protocol version to be used
     --iconv=CONVERT_SPEC    request charset conversion of filenames
     --checksum-seed=NUM     set block/file checksum seed (advanced)
     --noatime               do not alter atime when opening source files
 -4, --ipv4                  prefer IPv4
 -6, --ipv6                  prefer IPv6
     --version               print version number
(-h) --help                  show this help (-h is --help only if used alone)

Use "rsync --daemon --help" to see the daemon-mode command-line options.
Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation.
See http://rsync.samba.org/ for updates, bug reports, and answers
rsync error: syntax or usage error (code 1) at options.c(2369) [server=3.1.3]
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(235) [Receiver=3.1.3]
Error in rsync process.
Cleaning SSH master...

small adjustment in README

Hi Maurice,
maybe you could add this small adjustment for users in README:
when connecting to remote host instead:
./bsync ALICE_DIR bob@sshserver:BOB_DIR

add port number
./bsync "-p 4000" ALICE_DIR bob@sshserver:BOB_DIR

I read the code but it was not clear for me where to put the port number.
Excellent script and very usefull.
Bob

Symlinks aren't followed

I have a directory with a lot of symlinks (both files and dirs), and, when I sync it, the expected behavior would be for the actual file or directory to be synced, not the symlink itself. rsync has the -L flag which does exactly this, so I hope this wouldn't be too hard to implement:
In the same way that bsync has the -o flag to pass arguments to ssh, I propose for there to be a similar flag that allows us to pass custom arguments to rsync. However, this would require some further changes: for instance, to the code that generates the .bsync-snap to account for when we want to follow symlinks (pass the -L flag to find).
However, I also had to edit the flags that are currently passed to rsync by default, namely -a which conflicts with -L. According to man rsync, -a is the same as -rlptgoD, so I just changed -a to -rLptgoD to test, and the rsync part worked.

Paths to find , gfind and rsync binaries are hard coded in the script, missing -printf option

First let me thank you for putting together this script as there are very few tools for devices with embed Linux OS like NAS or Openwrt routers which don't have sufficient resources as normal Linux distros .
I gave a try to sync my NAS with my openwrt router which has external disk connected to it.
Immediately I notice that I bumped in the issue of hard coded binary references in the script.
As on my NAS where the script is running the paths are the following ones:

Linux 2.6.12.6-VENUS mips GNU/Linux

/tmp/usbmounts/sda1/root # which python3
/opt/bin/python3
/tmp/usbmounts/sda1/root # which find
/opt/bin/find
/tmp/usbmounts/sda1/root # which rsync
/opt/bin/rsync

My router with Opewrt Barrier Braker on it:
Linux 3.10.49 mips GNU/Linux

root@DIR:~# which rsync
/usr/bin/rsync

root@DIR:~# which find
/usr/bin/find

Resulting with EM: remote find not found

I think it would be a nice approach to modify the script so that it runs:
which find
which rsync
which gfind
and use the results to assign the full paths for this binaries to the corresponding variables
or if they don't exist to throw the proper EM to ask user to install namely the missing package(s).
I also did notice that the
find provided by the openwrt busybox doesn't recognize the parameter -printf

find: unrecognized: -printf
BusyBox v1.22.1 (2015-03-28 06:57:02 CET) multi-call binary.

Usage: find [-HL] [PATH]... [OPTIONS] [ACTIONS]

Search for files and perform actions on them.
First failed action stops processing of current file.
Defaults: PATH is current directory, action is '-print'

    -L,-follow      Follow symlinks
    -H              ...on command line only
    -xdev           Don't descend directories on other filesystems
    -maxdepth N     Descend at most N levels. -maxdepth 0 applies
                    actions to command line arguments only
    -mindepth N     Don't act on first N levels
    -depth          Act on directory *after* traversing it

Actions:
( ACTIONS ) Group actions for -o / -a
! ACT Invert ACT's success/failure
ACT1 [-a] ACT2 If ACT1 fails, stop, else do ACT2
ACT1 -o ACT2 If ACT1 succeeds, stop, else do ACT2
Note: -a has higher priority than -o
-name PATTERN Match file name (w/o directory name) to PATTERN
-iname PATTERN Case insensitive -name
-path PATTERN Match path to PATTERN
-ipath PATTERN Case insensitive -path
-regex PATTERN Match path to regex PATTERN
-type X File type is X (one of: f,d,l,b,c,...)
-perm MASK At least one mask bit (+MASK), all bits (-MASK),
or exactly MASK bits are set in file's mode
-mtime DAYS mtime is greater than (+N), less than (-N),
or exactly N days in the past
-user NAME/ID File is owned by given user
-group NAME/ID File is owned by given group
-size N[bck] File size is N (c:bytes,k:kbytes,b:512 bytes(def.))
+/-N: file size is bigger/smaller than N
-prune If current file is directory, don't descend into it
If none of the following actions is specified, -print is assumed
-print Print file name
-print0 Print file name, NUL terminated
-exec CMD ARG ; Run CMD with all instances of {} replaced by
file name. Fails if CMD exits with nonzero

But openwrt has separate command

root@DIR:~# printf
ash: usage: printf FORMAT [ARGUMENT...]

Any way to do the same task via it?
Thanks you.

bsync only supports filenames encoded in the current locale

Problem can not be clearly understood:

ceremcem@cca-erik:/media/ceremcem/zfs_zencefil$ bsync ./-.-\ cca-home+/ /media/ceremcem/7c32ca7d-fb9e-4c0b-8e89-6de320908b8b/cca-sync/-.-\ cca-home+/
Loading filelists...
Old filelist not found. Starting with empty history.
Traceback (most recent call last):
  File "/home/ceremcem/bin/bsync", line 779, in <module>
    dir2 = load_dir(ssh2, dir2name, ignores)
  File "/home/ceremcem/bin/bsync", line 403, in load_dir
    record = read_file_record(gen)
  File "/home/ceremcem/bin/bsync", line 307, in read_file_record
    i,p,t,d,s,perms = next(gen),next(gen),next(gen),next(gen),next(gen),next(gen)
  File "/home/ceremcem/bin/bsync", line 294, in fileLineIter
    charsJustRead = inputFile.read(readSize)
  File "/usr/lib/python3.3/codecs.py", line 313, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 1277: invalid start byte

move with ssh

Hello,
I guess that line 634 there is a missing test: moves!=[]
If not move operation are impossible via ssh.
E.

Not working in TCSH

The code does not work "out-of-the-box" on TCSH (both local and server):

python bsync user@host:/path/2/stuff

Gives the following error/response:

cd: Too many arguments.
Error: could not open directory: user@host:/path/2/stuff (is it created?)

What is "checked" by -c?

Hi,
I saw on row 765 of bsync that there is a -c parameter who means "check".
It is not display on help menu and i don't know what it means, can you explain us?
Thanks

invalid literal

Was testing the most recent version and got the following error:

Traceback (most recent call last):
  File "/home/reagle/bin/bsync", line 866, in <module>
    show_conflict(f1, f2, path)
  File "/home/reagle/bin/bsync", line 479, in show_conflict
    p1 = getfilemode(f1.type, f1.perms)+" "+f1.size+"B ("+getdatestr(f1)+")"
  File "/home/reagle/bin/bsync", line 469, in getfilemode
    return type + stat.filemode(int(perms, 8))[1:]
ValueError: invalid literal for int() with base 8: ''

Sync without interactive prompt

Hi dooblem,
I'm Sorry, this is not a issue but I don´t find other method to contact to you.
I have one question, is it possible synchronised directories without interactive prompt?.
I want to use the cron daemon to automate the synchronization of a directory, but this will not be possible if I have an interactive prompt.

For example:

[root@aest-segu42 apeiti]# /root/bsync -v /home/apeiti/ [email protected]:/home/apeiti

Password:

Loading filelists...
Loading original filelist from snap files...
Loading .bsync-snap-20140616160929.950125...
Loading dir1 filelist...
Loading dir2 filelist...
Analysing original paths...
Analysing remaining new paths in dir1...
Analysing remaining new paths in dir2...

(LEFT DIR CONTENT) (RIGHT DIR CONTENT) (ACTION)

                                  <--  A000552.dmp                           (copy)

passwd_acceso1 <-- passwd_acceso1 (sync)

Todo in /home/apeiti/: cp:1 sync:1
Todo in [email protected]:/home/apeiti:
Apply actions? [y/N] y

Applying actions...
Applying actions in dir2...
Applying actions in dir1...
rsync: A000552.dmp
rsync: passwd_acceso1
Updating filelists...
Updating snap files: .bsync-snap-20140616161840.452684...
Done!
Cleaning SSH master...
[root@aest-segu42 apeiti]#

Thanks a lot

SyntaxError: invalid syntax

I am getting the following error when try to run it:

[root@linux-utils test]# ./bsync
File "./bsync", line 97
print(s, file=sys.stderr)
^
SyntaxError: invalid syntax

[root@linux-utils test]# python -V
Python 2.6.6

[root@linux-utils test]# uname -a
Linux linux-utils 2.6.32-504.16.2.el6.x86_64 #1 SMP Tue Mar 10 17:01:00 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.