shodanshok/blocksync

blocksync's Issues

support stdin/stdout ?

would it be possible to have support for stdin/stdout with blocksync ?

using "-" as filename does not seem to work

this would be useful, because for example with borg backup you could avoid fuse mounting the borg archive to access the archive and use borg extract directly, for example

borg extract --stdout /mnt/repo1::sda_07_10_2022 | blocksync.py - localhost /dev/sdc

python2: can't open file 'blocksync.py': [Errno 2] No such file or directory

btw, i'm getting the error below and could workaround it by removing "python2" from the command

before

 cmd = ['python2', '/root/blocksync.py', 'server', dstpath, '-a', options.hashalg,

after

    cmd = ['blocksync.py', 'server', dstpath, '-a', options.hashalg,    

Errors:

# borg extract --stdout /mnt/repo2::sda_07_10_2022_8 |blocksync.py - localhost /dev/sdc

Dry run     : False
Local       : True
Block size  : 128.0 KB
Skipped     : 0 blocks
Hash alg    : sha512
Crypto alg  : aes128-cbc
Compression : False
Read cache  : True
SRC command : /usr/local/bin/blocksync.py - localhost /dev/sdc
DST command : python2 blocksync.py server /dev/sdc -a sha512 -b 131072 -k 0

python2: can't open file 'blocksync.py': [Errno 2] No such file or directory
ERROR: connecting to or invoking blocksync on the remote host!

# python2 blocksync.py
python2: can't open file 'blocksync.py': [Errno 2] No such file or directory

Windows is support?

hello，i want to use this tool for vm migration. vm os is Windows. Can it support for Windows? Besides, this tool params is support bwlimit and Incremental sync? thanks~

blocksync.py python3 bug

i'm getting this when using python3 version of blocksync. pyhton2 version works

/root/borg extract --stdout /backup/test::second | mbuffer -q | /root/blocksync.py - localhost /dev/sdc

Dry run : False
Local : True
Block size : 128.0 KB
Skipped : 0 blocks
Hash alg : sha512
Crypto alg : aes128-cbc
Compression : False
Read cache : True
SRC command : /root/blocksync.py - localhost /dev/sdc
DST command : /root/blocksync.py server /dev/sdc -a sha512 -b 131072 -k 0

Synching...
Traceback (most recent call last):
File "/root/blocksync.py", line 386, in
sync(srcpath, dsthost, dstpath)
File "/root/blocksync.py", line 246, in sync
for (l_block, l_sum) in getblocks(f):
File "/root/blocksync.py", line 110, in getblocks
block = f.read(options.blocksize)
File "/usr/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 0: invalid continuation byte
Traceback (most recent call last):
File "/root/blocksync.py", line 378, in
server(dstpath)
File "/root/blocksync.py", line 149, in server
res, complen = in_line.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
mbuffer: error: outputThread: error writing to at offset 0x20000: Broken pipe
mbuffer: warning: error during output to : Broken pipe
Local Exception
Traceback (most recent call last):
File "borg/archiver.py", line 5168, in main
File "borg/archiver.py", line 5099, in run
File "borg/archiver.py", line 183, in wrapper
File "borg/archiver.py", line 198, in wrapper
File "borg/archiver.py", line 917, in do_extract
File "borg/archive.py", line 781, in extract_item
BrokenPipeError: [Errno 32] Broken pipe

Platform: Linux debian 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64
Linux: Unknown Linux
Borg: 1.2.3 Python: CPython 3.9.16 msgpack: 1.0.4 fuse: llfuse 1.4.2 [pyfuse3,llfuse]
PID: 1711 CWD: /root
sys.argv: ['/root/borg', 'extract', '--stdout', '/backup/test::second']
SSH_ORIGINAL_COMMAND: None

Python2 EOL

Python2 is EOL for a while ( https://www.python.org/doc/sunset-python-2/ ) , what about porting blocksync to Pyhton3 ? (did not have a look yet, how difficult that would be)