rmayr/dvcs-autosync

dvcs-autosync is a project to create an open source replacement for Dropbox/Wuala/Box.net/etc. based on distributed version control systems (DVCS). It offers nearly instantaneous mutual updates when a file is added or changes on one side but with the added benefit of (local, distributed) versioning and does not rely on a centralized service provider.

dvcs-autosync's Issues

syncmethod = none should not raise an error notification

#9 was recently solved (thanks for that!).

Minor annoyance, with syncmethod=none an error notification is raised. On some WMs (awesome in my case) notifications marked urgent remain there until dealt with. Is it possible to split the notification between the supported 'syncmethod=none' case and the unsupported 'syncmethod=a_wrong_option'?

Consider this a minor report.

dvcs-autosync incorrectly records changes on a file `4913`

Here's my config (pretty much default)
cat .config/autosync/pim
[autosync]
path = ~/.local/share/pim
pidfile = ~/.cache/autosync.pid
syncmethod = xmpp
pulllock = conservative
readfrequency = 5
ignorepath = .git .svn .hg

[dvcs]
statuscmd = git status | grep -iq "nothing to commit"
addcmd = git add %s
rmcmd = git rm %s
modifycmd = git add %s
movecmd = git rm %s 
    git add %s
startupcmd = git add -A
commitcmd = git commit -m "Autocommit"
pushcmd = git push
pullcmd = git pull
remoteurlcmd = git config --get remote.origin.url

[xmpp]
username = [email protected]
password = mypass

notice that in the target directory ~/.local/share/pim, there is no file 4913, nor any file containing the string 4913 or whatever:
$ find ~/.local/share/pim -name '4913'
$ grep -R 4913 ~/.local/share/pim
$

Yet, I have it quite often, when I change any file, dvcs-autosync detects a change on the file 4913, tries to commit it, which obviously fails. Here is the output of an autosync session where I 1) start the program (with the config above), remove 12 lines from the file dvcs-autosync in the watched directory.

http://pastie.org/1778450
Interestingly, it receives a coalesce event for the file dvcs-autosync, then for the file 4913 then again for dvcs-autosync

In FileChangeHandler::_filter_and_handle_actions(self, args) the args variable for the relevent call is this:
DEBUG:root:['/home/dieter/.local/share/pim/4913', ['/home/dieter/.local/share/pim/4913']]

installed software:
pyinotify 0.9.1-1
python2 2.7.1-7
I have this issue with both the develop and master branches, in fact I always had this issue with either version.

jabberbot ignores messages from other clients

When I follow the setup instructions from the website, and run the clients on 2 pc's, synchronisation doesn't work, because they ignore each others xmpp messages.
Example:

DEBUG:root:Git reported that there is nothing to commit, not touching commit timer
INFO:jabberbot:*** roster ***
INFO:jabberbot:  [email protected]
INFO:jabberbot:*** roster ***
DEBUG:jabberbot:*** props = [u'jabber:client']
DEBUG:jabberbot:*** jid = [email protected]/AutosyncJabberBot on dieter-ws
DEBUG:jabberbot:*** username = s
DEBUG:jabberbot:*** type = chat
DEBUG:jabberbot:*** text = pushed dieter.plaetinck.be:.local/share/pim.git

INFO:jabberbot:Ignoring message from unseen guest: [email protected]/AutosyncJabberBot on dieter-ws
DEBUG:jabberbot:I've seen: []
DEBUG:jabberbot:*** props = [u'jabber:client']
DEBUG:jabberbot:*** jid = [email protected]/AutosyncJabberBot on dieter-ws
DEBUG:jabberbot:*** username = s
DEBUG:jabberbot:*** type = chat
DEBUG:jabberbot:*** text = pushed dieter.plaetinck.be:.local/share/pim.git

INFO:jabberbot:Ignoring message from unseen guest: [email protected]/AutosyncJabberBot on dieter-ws
DEBUG:jabberbot:I've seen: []
DEBUG:jabberbot:*** props = [u'jabber:client']
DEBUG:jabberbot:*** jid = [email protected]/AutosyncJabberBot on dieter-ws
DEBUG:jabberbot:*** username = s
DEBUG:jabberbot:*** type = chat
DEBUG:jabberbot:*** text = pushed dieter.plaetinck.be:.local/share/pim.git

INFO:jabberbot:Ignoring message from unseen guest: [email protected]/AutosyncJabberBot on dieter-ws
DEBUG:jabberbot:I've seen: []

there should be a "safetycheckcmd" to avoid trying to commit in some cases

Specifically, the case where a merge conflict occurs. In this case the user needs to resolve the conflict manually first (which causes a bunch of file writes/deletes/updates), but dvcs-autosync may only do its job after the version control system says it's safe to do so (i.e. the conflict has been resolved)
for this we could introduce something like a safetycheckcmd configuration variable, this command could inspect the .git folder (although I couldn't figure out where it stores the information that a directory is currently in conflict) before performing any other action.

Use of logger in stead of logging causes crash on improper configuration

Trivial issue (unless I'm overlooking something here) causing a crash on using notifymethod = all without alsonotify set.

Line 900 issues a warning using logger.warning, which should be logging.warning.

https://github.com/rmayr/dvcs-autosync/blob/master/dvcs-autosync#L900

decide on which bugtracker to use

I have a whole bunch of tickets I want to file, so once you figure out which bugtracker we'll use, I can start filing :)

dvcs-autosync push instead of pull

Hello.
Here's my config:

[autosync]
path = ~/autosync
syncmethod = xmpp
notifymethod = all
pulllock = conservative
readfrequency = 5
ignorepath = .git .svn .hg src/packages src/java/openuat 
    src/csharp/sparkleshare src/cpp/cross/keepassx src/android/ipv6config 

[dvcs]
statuscmd = git status | grep -iq "nothing to commit"
addcmd = git add %s
rmcmd = git rm -r %s
modifycmd = git add %s
movecmd = git rm %s 
    git add %s
startupcmd = git add -A
commitcmd = git commit -m %s
pushcmd = git push
pullcmd = git pull
remoteurlcmd = git config --get remote.origin.url

[xmpp]
username = ***
password = ***

I have two machines, Solaria and Gaia, which are accessible with ssh without authentification.
If I execute git pull then git push on my side, it works, but if I touch something on one of the machines, I get that on the other:

2012-07-14 10:15:00,275:INFO:root:NOTIFICATION: Pulling changes: Pulling changes from remote repository
2012-07-14 10:15:00,505:WARNING:root:NOTIFICATION: Command failed: Command 'git push' in '/home/giskard/autosync' failed.  Output:
To solaria:autosync.git
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'solaria:autosync.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

I guess there is something wrong, but I don't find anything.
I use the AUR package dvcs-autosync-git on archlinux.

loglevel should be configurable

it should be up to users whether they want date/time (and which format) in the messages, and which loglevel they want. this could be made configurable through the config file

show warning/error notifications when commands fail

when any command execution fails (like a git push or whatever), show warning to the user.
I can fix this easily and do a pull request, this ticket is mostly to remind myself.

ImportError: No module named jabberbot

Traceback (most recent call last):
  File "/usr/bin/dvcs-autosync", line 32, in <module>
    import jabberbot, xmpp
ImportError: No module named jabberbot

the reason is:

    sys.path.insert(0, '/usr/share/dvcs-autosync')
    import jabberbot, xmpp

this is clearly broken.
in my case jabberbot.py is here (after a python2 setup.py install):

/usr/lib/python2.7/site-packages/dvcsautosync/jabberbot.py

Website links to wrong repo and issue tracker

https://www.mayrhofer.eu.org/dvcs-autosync

has dead links to repo and issue tracker. I expect them to link both here.

Is this project abandoned?

Are there any plans for improvements or new features?

Add syncmethod = none

Hi, syncmethod = none would be useful for automated git-based backup of every little change in the repository.

My use-case is that I have a folder containing song lyrics, bind-mounted from my Dropbox (where its shared with windows users). They may make changes at any time, as those changes come in from Dropbox dvcs-autosync commits them, so I can rely on the simple git mechanisms rather than dropbox's rollbacks (which are confusing to me).

For this usecase I'm not informing anyone when I make a file change, so I don't use xmpp (and don't set it up). There's error messages as the jabberbot dies each time, but everything seems to work fine. So a syncmethod = none which just didn't bother with all that stuff would seem to work fine as well.

multiple autosync.py instances can run on the same directory

I can run autosync.py <path to config> twice.
Both initialize, do the xmpp login, initial pull etc, but then you get (on both):

 WARNING:root:AutosyncJabberbot:Received IOError while trying to handle incoming messages, trying to reconnect now

You can then kill one of them, and the remaining one will keep working normally.
(I didn't dare to test what happens when you modify files in the git repository, but likely both instances will try to commit it at the same time)

autosync.py should somehow detect another instance is already running on the same directory/config and abort. optionally, a -k flag to kill all instances and -k <config> would kill any other instance running for the given config.

obviously, multiple instances should be able to run as long as they have a different config file.
Actually, we should check for a different repository, but different config is likely easier to implement and should do the job. (why would a user have different configs pointing to the same repository?)

Only last change after $readfrequency gets synced

If readfrequency is set to a high value (e.g. 60), all but the last changes get ignored on sync.

How to reproduce:

• set readfrequency to something big, like e.g. 60 seconds
• cd to sync path
• create file1 --> sync count-down starts
• wait half a minute or so, then create file2 --> sync count-down restarts as it should
• wait for sync; ssh to a synced remote client and check contents of sync path folder
• file1 is missing over there...

It seems, that upon later changes, the missing files will get synced from time to time, but I don't know how reliably, though.

This behavior is also present for shorter intervals (e.g. 5 sec) and leads to phenomena like editor's swap files (e.g. of vim editor) get synced on opening of a text file, but won't get deleted afterwards, as the last file modified is the text file itself and not the swap file.

Comment:
I like the lean approach of dvcs-autosync over the mono-tainted sparkleshare, but this bug is a real issue in my eyes, as I cannot seem to rely on the data integrity as it is...

cannot launch again if pid written in .pid file is used by another process

The program thinks autosync is launched with that pid even if it is taken by another process

SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

After a few days of running dvcs-autosync it errors out with the following error message. It is not syncing files at the crash. Just idling.

Error:

ERROR:root:Error pinging the server: Ping timeout, treating as ping timeout.
WARNING:jabberbot:Received IOError while trying to handle incoming messages, trying to reconnect now
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/bin/dvcs-autosync", line 227, in _process_thread
    self.conn = self.connect()
  File "/usr/share/dvcs-autosync/jabberbot.py", line 140, in connect
    conres = conn.connect()
  File "build/bdist.macosx-10.9-intel/egg/xmpp/client.py", line 205, in connect
    while not self.TLS.starttls and self.Process(1): pass
  File "build/bdist.macosx-10.9-intel/egg/xmpp/dispatcher.py", line 303, in dispatch
    handler['func'](session,stanza)
  File "build/bdist.macosx-10.9-intel/egg/xmpp/transports.py", line 337, in StartTLSHandler
    self._startSSL()
  File "build/bdist.macosx-10.9-intel/egg/xmpp/transports.py", line 315, in _startSSL
    tcpsock._sslObj    = socket.ssl(tcpsock._sock, None, None)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 64, in ssl
    return _realssl.sslwrap_simple(sock, keyfile, certfile)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 477, in sslwrap_simple
    ssl_sock.do_handshake()
SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

Running on OS X 10.9. Using jabber.chaotic.de as XMPP service.