skarra/asynk

Flexible Bi-directional sync for Emacs BBDB, Google Contacts, MS Outlook, MS Exchange, CardDAV

Created : Wed Aug 24 22:41:42 IST 2011

This is only a short introduction and some key points of interest. For more details on how to use ASynK please refer to the included documentation, or the official website for ASynK is at: http://asynk.io/ You will find links to latest download, complete documentation, and other information on that website.

About

ASynK - Awesome Sync (by Karra), is a flexible Contacts synchronization platform written in Python. High level features include:

• ASynK allows two-way sync of Contacts across any two supported sources - Google Contacts, Microsoft Outlook, Microsoft Exchange Server, any CardDAV server and Emacs BBDB.
• You can use ASynK to copy contacts from one Google account to another (say from a Google Apps account at work to a personal address before you leave your job). You can do this at the level of folders (see below)
• ASynK is the only two-way synchronization option for BBDB known to be in existence today. So if you are keen to keep your BBDB content synchronized with a mobile device or Outlook, look no further.
• ASynK works on Windows, MacOS X, and Linux. Outlook synchronization only works on Windows.
• Synchronization is granular to the level of ‘Folders’, i.e. you can synchronize one folder of Google Contacts with a Outlook folder (on Windows at work, say), and another Google folder in the same account with BBDB (On your Mac, say)
• The infrastructure is there to add more database and item types (i.e. tasks, notes, etc.), but currently only Contacts synchronization is supported.
• It is free software, and released under the GNU AGPL (Affero GPL) version 3

Installation

Fresh Installation

As you are reading this file, the most efficient installation is to recursively clone this git repository to your machine. Then see the section titled Usage below to invoke ASynK.

git clone  --recursive https://github.com/skarra/ASynK.git

Updating an earlier install

If you are updating your ASynK installation from an earlier version, please note that many new git-submodules are added to the project. So for your update to work properly you need to update all sub-modules. Recommended way to do this would be to fun the following command from the root of your ASynK directory, after you pull in the latest changes

git submodule update --init --recursive

Dependencies

ASynK is completely written in Python. For most use cases and set ups you only need to have Python 2.x (x >= 7) installed. Some additional dependencies include:

• If you wish to sync from/to MS Outlook:
  • MS Windows
  • Python 2.x (2.7 or later) for Windows
  • MS Outlook
  • Pywin32 from Mark Hammond, available from http://sourceforge.net/projects/pywin32/
• On Debian (and likely Ubuntu) you will need the following packages: sudo apt-get install python-argparse
• The following python packages.
  • dateutil
  • httplib2
  • oauth2client
  • PyOpenSSL
  • google-api-python-client

  You could install all of them with just

$ pip install -r requirements.txt

Usage

The recommended way to use ASynK is to first create your own ‘sync profile’ which specifies which folders and PIM DBs you want to keep in sync. Refer to the documentation for more details.

For the really impatient, ASynK comes with two default sync profiles that kick in if no other profile is configured.

$ python asynk.py --op=sync [--dry-run] [--log=debug]

If you run the above command on Windows, the entire default outlook contacts folder will get synched to “My Contacts” on Google Contacts. (You will be prompted for your google username and password).

On any non-windows platform if you have a ~/.bbdb will simply be “backed up” to /tmp/asynk.bbdb.

For further usage help, try “python asynk.py -h”

asynk's Issues

Only load and process stores / folders that matter

This applies specifically to Outlook. For many profile and folder operations, we are typically interested in only one folder. The startup code flow needlessly iterates through and tries to make sense of all the folders. This increases application startup time considerably. We should avoid doing all this unnecessary work at startup.

Add Support for baikal carddav server

Hi,

nice tool, but i cant get it to run with baikal server.

Error : Cant find adressbook.

I use the same addressbook link which works in other apps.

with best daniel

Browser based UI

v0.1.0 had a wxPython based UI. v0.2.x has removed all that code, and only supports a command line interface on both Unix and Windows. We really need a good GUI to boost adoption on windows platform.

I am tending towards embedding the tornado server and implement a browser based UI. Advantages are: code as well as the UX will be consistent across all platforms. Main complexity is with regards to use of WebSockets - an area I am not really familiar at this time.

ASynK should gracefully deal with vCard parsing issues

Bugs in the vCard file formats on server side should not throw ASynK into a spin. We should handle them gracefully - skipping invalid entries and continue with rest of the stuff.

Want to solve this The Right Way - i.e. the dry-run statistics should themselves include a category 'Errors' (i.e. in addition to New, Modified, etc.) and when there are errors, the sync timestamp will not be updated.

Log and backup files need to be purged periodically

Log and bbdb backup files older than some specified period (a config.json parameter) should be deleted.

autoconf fails

I checked out a copy on a Debian Squeeze desktop.

marajade-[/sandel/src/ASynK] mcr 10032 %autoreconf
configure.ac:9: required file `./install-sh' not found
configure.ac:9:   `automake --add-missing' can install `install-sh'
configure.ac:9: required file `./missing' not found  
configure.ac:9:   `automake --add-missing' can install `missing' 
doc/Makefile.am:9: variable `DIST_SOURCES' is defined but no program or
doc/Makefile.am:9: library has `DIST' as canonical name (possible typo)
autoreconf: automake failed with exit status: 1

Although kinda crappy, I suggest checking in the configure script, as autoconf is very very very brittle.

PDF link does not work.

http://asynk.io/doc/asynk.pdf is 404

Quotation marks need to be escaped in BBDB

I have two addresses in my Google Contacts that have quotation marks in the address field. The quotation marks are synched to BBDB without being escaped, leading to a parse error.

Need simple default profiles to get people started with minimal fiddling

It is quite clear that the most common use case for ASynK will be one of the following:

(a) sync the default outlook contacts folder to the 'My Contacts' folder on google
(b) sync the full BBDB database from ~/.bbdb to the 'My Contacts' folder on google

We should have two simple sample profiles that should capture those two scenarios and be available "out of the box". Once the proposed change is implemented, a user should be able to perform a sync with a single command immediately after fetching ASynK:

$ python asynk.py --op=sync

and the program should ask for the google user name - sync using the bbgc profile if on Unix, and olgc profile if on Windows. That will allow people to get started without digging through the docs, which admittedly can be a bit of a chore.

AsynkError not imported from state_collection.py

Traceback (most recent call last):
  File "./asynk.py", line 302, in <module>
    main()
  File "./asynk.py", line 299, in main
    asynk.dispatch()
  File "/home/cassou/Downloads/asynk/asynk/asynk_core.py", line 97, in dispatch
    res = getattr(self, self.get_op())()
  File "/home/cassou/Downloads/asynk/asynk/asynk_core.py", line 366, in op_sync
    pname = self._load_profile()
  File "/home/cassou/Downloads/asynk/asynk/asynk_core.py", line 584, in _load_profile
    self._login()
  File "/home/cassou/Downloads/asynk/asynk/asynk_core.py", line 85, in _login
    coll.login()
  File "/home/cassou/Downloads/asynk/asynk/state_collection.py", line 291, in login
    raise AsynkError("%s: Cannot use any CardDAV accounts" % e)
NameError: global name 'AsynkError' is not defined

UnboundLocalError: local variable 'bbfn' referenced before assignment

As reported by Sam Steingold on the ASynK mailing list on Nov 27, 2012

$ python asynk.py --op=sync --db bb --op=list-folders
[16:25:55.543     INFO] Debug logging to file: /home/sds/.asynk/logs/asynk_logs.2012-11-26.16-25-55.542977
[16:25:55.543     INFO] Deleting log files older than 7 days, if any...
[16:25:55.543     INFO] Deleting log files older than 7 days, if any...done
Traceback (most recent call last):
  File "asynk.py", line 890, in <module>
    main()
  File "asynk.py", line 110, in main
    asynk.dispatch()
  File "asynk.py", line 411, in dispatch
    res = getattr(self, self.get_op())()
  File "asynk.py", line 426, in op_list_folders
    self._login()
  File "asynk.py", line 304, in _login
    self.set_db(db1id, getattr(self, login_func)())
  File "asynk.py", line 827, in login_bb
    if not bbfn:
UnboundLocalError: local variable 'bbfn' referenced before assignment

Ensure phone labels are not lost when saving to Outlook

Outlook / EWS do not really support custom labels for phones and email addresses. If we are getting such a thing from another source ensure it is saved to a custom property and restored properly. This has been implemented for Exchange store already (in the branch 'exchange' as of this time)

Have a single PropTags() instance

In the current implementation, each outlook folder instantiates its own PropTags() object. It is meant to bea singleton class. We should rewrite some of the code to make it a singleton, will save startup time and memory.

KeyError: 'dry_run' on first run

I'm trying to launch ASynK (2.0.0-rc2) for the first time. Here is the output I get:

$ ./result/bin/asynk.py 
Applying base config from file config/config_v7.json...
Applying base config from file config/config_v7.json...done
Applying user customizations from file /home/cassou/.asynk/config.py...
Applying user customizations from file /home/cassou/.asynk/config.py...done
[15:15:09.256     INFO] Debug logging to file: /home/cassou/.asynk/logs/asynk_logs.2015-01-22.15-15-09.256145
Traceback (most recent call last):
  File "./result/bin/asynk.py", line 1125, in <module>
    main()
  File "./result/bin/asynk.py", line 80, in main
    asynk.dispatch()
  File "./result/bin/asynk.py", line 508, in dispatch
    if not self.is_dry_run():
  File "./result/bin/asynk.py", line 903, in is_dry_run
    return self._get_att('dry_run')
  File "./result/bin/asynk.py", line 858, in _get_att
    return self.atts[att]
KeyError: 'dry_run'

barfs on names with commas in them

In my owncloud contacts list, if I have a contact with a name like "Paul, that guy from downstairs", I get:

Traceback (most recent call last):
  File "./asynk.py", line 1026, in <module>
    main()
  File "./asynk.py", line 80, in main
    asynk.dispatch()
  File "./asynk.py", line 477, in dispatch
    res = getattr(self, self.get_op())()
  File "./asynk.py", line 715, in op_sync
    sync.prep_lists(self.get_sync_dir())
  File "/home/me/src/ASynK/asynk/sync.py", line 283, in prep_lists
    return self._prep_lists_2_way(self.get_f1(), self.get_f2())
  File "/home/me/src/ASynK/asynk/sync.py", line 179, in _prep_lists_2_way
    f2.prep_sync_lists(f1.get_dbid(), f2sl)
  File "/home/me/src/ASynK/asynk/folder_cd.py", line 66, in prep_sync_lists
    curi  = self.get_itemids(pname, destid)
  File "/home/me/src/ASynK/asynk/folder_cd.py", line 119, in get_itemids
    self._refresh_contacts()
  File "/home/me/src/ASynK/asynk/folder_cd.py", line 285, in _refresh_contacts
    cons  = self.find_items(hrefs)
  File "/home/me/src/ASynK/asynk/folder_cd.py", line 165, in find_items
    cd = CDContact(self, vco=vco, itemid=key)
  File "/home/me/src/ASynK/asynk/contact_cd.py", line 115, in __init__
    self.init_props_from_vco(vco)
  File "/home/me/src/ASynK/asynk/contact_cd.py", line 182, in init_props_from_vco
    self._snarf_names_gender_from_vco(vco)
  File "/home/me/src/ASynK/asynk/contact_cd.py", line 231, in _snarf_names_gender_from_vco
    self.set_firstname(vco.n.value.given)
  File "/home/me/src/ASynK/asynk/contact.py", line 98, in set_firstname
    self._set_prop('firstname', string.strip(val) if val else None)
  File "/usr/lib/python2.7/string.py", line 257, in strip
    return s.strip(chars)
AttributeError: 'list' object has no attribute 'strip'

Entering

    import sys
    print >>sys.stderr, val

into that function, I get:

[…]
Raspberry Pi
Vlastimil
[u'Paul\\', u' that guy from']
Traceback (most recent call last):
  File "./asynk.py", line 1026, in <module>
[…]

so it seems like something somewhere interprets the , wrongly?

What CardDAV URL to use with Owncloud?

I could not figure out from the docs what exact CardDAV URL I should be sending. When Owncloud says my url is https://localhost/owncloud/remote.php/carddav/addressbooks/me/contacts, what URL should I be using? I tried a bunch of substrings of it, here are the error codes I get:

https://localhost/owncloud/remote.php/carddav/addressbooks/me/contacts
Could not open connection to Carddav Server (https://localhost). Error: Code: 404, Reason: Not Found
--
https://localhost/owncloud/remote.php/carddav/addressbooks/me/
Could not open connection to Carddav Server (https://localhost). Error: Code: 404, Reason: Not Found
--
https://localhost/owncloud/remote.php/carddav/addressbooks/me
Could not open connection to Carddav Server (https://localhost). Error: Code: 404, Reason: Not Found
--
https://localhost/owncloud/remote.php/carddav/addressbooks/
Found current principal path: /owncloud/remote.php/carddav/principals/me/
Could not open connection to Carddav Server (https://localhost). Error: Code: 500, Reason: Internal Server Error
--
https://localhost/owncloud/remote.php/carddav/addressbooks
Found current principal path: /owncloud/remote.php/carddav/principals/me/
Could not open connection to Carddav Server (https://localhost). Error: Code: 500, Reason: Internal Server Error
--
https://localhost/owncloud/remote.php/carddav/
Found current principal path: /owncloud/remote.php/carddav/principals/me/
Could not open connection to Carddav Server (https://localhost). Error: Code: 500, Reason: Internal Server Error
--
https://localhost/owncloud/remote.php/carddav
Found current principal path: /owncloud/remote.php/carddav/principals/me/
Could not open connection to Carddav Server (https://localhost). Error: Code: 500, Reason: Internal Server Error
--
https://localhost/owncloud/remote.php/
Could not open connection to Carddav Server (https://localhost). Error: Code: 404, Reason: Not Found
--
https://localhost/owncloud/remote.php
Could not open connection to Carddav Server (https://localhost). Error: Code: 404, Reason: Not Found
--
https://localhost:443/owncloud
Could not open connection to Carddav Server (https://localhost:443). Error: Code: 301, Reason: Moved Permanently
--
https://localhost:443/
Could not open connection to Carddav Server (https://localhost:443). Error: Code: 405, Reason: Method Not Allowed
--
https://localhost:443
Could not open connection to Carddav Server (https://localhost:443). Error: Code: 405, Reason: Method Not Allowed

Unable to extract first and last names on Google Contacts

Hi!

I am trying ASynK with my Google contacts but I am unable to get a BBDB file with first and last names. I get the email address correctly, but first and last names are always nil despite being correctly present on the web interface.

Unable to sync with OwnCloud7

On master @ 4c602a4 having trouble synching with oc7. Others, in particular @rrix are able to sync; I am not sure how. This is a possible bug in ASynK, but this works quite well with Apple CalendarServer, Baikal, and other such servers.

Logs are as below:

Srirams-MacBook-Pro:asynk.co sriramkarra$ python ./asynk.py --op=sync --user-dir=~/.asynk.cd --name=oc7 --dry-run --log=debug
Applying base config from file /Users/sriramkarra/stuff/code/python/asynk.co/config/config_v7.json...
Applying base config from file /Users/sriramkarra/stuff/code/python/asynk.co/config/config_v7.json...done
Applying user customizations from file /Users/sriramkarra/.asynk.cd/config.py...
Applying user customizations from file /Users/sriramkarra/.asynk.cd/config.py...done
[10:52:35.328     INFO] Debug logging to file: /Users/sriramkarra/.asynk.cd/logs/asynk_logs.2014-09-28.10-52-35.327743
[10:52:35.332     INFO] Parsing BBDB file /Users/sriramkarra/dotfiles/.bbdb.misc/bbdb.testbbcd.own...
[10:52:35.334    DEBUG] New BBContactsFolder: default
[10:52:35.334     INFO] Parsing BBDB Store with encoding utf-8...
[10:52:35.358     INFO] Parsing BBDB Store with encoding utf-8...Success
[10:52:35.359     INFO] Successfully parsed 9 entries.
[10:52:35.454     INFO] Logging into CardDAV server: http://cloud.fort.kickass.systems:10080/owncloud/remote.php/carddav/principals/skarra/ - user: skarra...
[10:52:35.455    DEBUG] PIMDB cd does not have email_domains.
[10:52:35.455    DEBUG] PIMDB cd does not have postal_map
[10:52:35.455    DEBUG] PIMDB cd does not have notes_map
[10:52:35.455    DEBUG] PIMDB cd does not have phones_map
[10:52:35.456    DEBUG] Sending request OPTIONS to server cloud.fort.kickass.systems:10080/owncloud/remote.php/carddav/principals/skarra/...
[10:52:36.733    DEBUG] Sending Request URL: /owncloud/remote.php/carddav/principals/skarra/    Method: OPTIONS
[10:52:37.335    DEBUG] Received Response: HTTP/1.1 401 Unauthorized

[10:52:37.335    DEBUG] Sending request OPTIONS to server cloud.fort.kickass.systems:10080/owncloud/remote.php/carddav/principals/skarra/...
[10:52:37.335    DEBUG] Sending Request URL: /owncloud/remote.php/carddav/principals/skarra/    Method: OPTIONS
[10:52:37.858    DEBUG] Received Response: HTTP/1.1 200 OK

[10:52:37.858    DEBUG] Sending Request URL: /owncloud/remote.php/carddav/principals/skarra/    Method: PROPFIND
[10:52:38.492    DEBUG] Received Response: HTTP/1.1 207 Multi-Status

Found current principal path: /owncloud/remote.php/carddav/principals/skarra/
[10:52:38.784    DEBUG] Sending Request URL: /owncloud/remote.php/carddav/principals/skarra/    Method: PROPFIND
[10:52:39.405    DEBUG] Received Response: HTTP/1.1 207 Multi-Status

[10:52:39.407    DEBUG] Found principal: skarra
[10:52:39.407    DEBUG] CDPIMDB.fetch_folders(): Begin
[10:52:39.407    DEBUG] Exploring following root paths: ['/owncloud/remote.php/carddav/addressbooks/skarra/']
[10:52:39.409    DEBUG] Processing root path /owncloud/remote.php/carddav/addressbooks/skarra/
[10:52:39.697    DEBUG] Sending Request URL: /owncloud/remote.php/carddav/addressbooks/skarra/  Method: PROPFIND
[10:52:40.293    DEBUG] Received Response: HTTP/1.1 207 Multi-Status

[10:52:40.293    DEBUG]   Found following properties: ['/owncloud/remote.php/carddav/addressbooks/skarra/'].
[10:52:40.293    DEBUG] CDPIMDB.fetch_folders(): Done.
[10:52:40.602    DEBUG] Sending Request URL: /owncloud/remote.php/carddav/addressbooks/skarra/  Method: PROPFIND
[10:52:41.221    DEBUG] Received Response: HTTP/1.1 207 Multi-Status

[10:52:41.222    DEBUG] Coud not find default adbk Property. Trying to use first of the available contacts folders
Traceback (most recent call last):
  File "./asynk.py", line 1125, in <module>
    main()
  File "./asynk.py", line 80, in main
    asynk.dispatch()
  File "./asynk.py", line 511, in dispatch
    res = getattr(self, self.get_op())()
  File "./asynk.py", line 775, in op_sync
    pname = self._load_profile()
  File "./asynk.py", line 1119, in _load_profile
    self._login()
  File "./asynk.py", line 312, in _login
    self.set_db(db2id, getattr(self, login_func)())
  File "./asynk.py", line 1061, in login_cd
    self.get_cduser(), self.get_cdpw())
  File "/Users/sriramkarra/stuff/code/python/asynk.co/asynk/pimdb_cd.py", line 57, in __init__
    self.set_def_folders()
  File "/Users/sriramkarra/stuff/code/python/asynk.co/asynk/pimdb_cd.py", line 135, in set_def_folders
    assert(len(fs) > 0)
AssertionError

owncloud carddav sync doesn't seem to swallow new contacts

this is with the latest master, 4e15e27.

my use-case: sync my google contacts -> bbdb (one way, seems to work as expected), and then a two way sync with my owncloud install (owncloud v6 rc4).

$ ./asynk.py --op=list-folders --db cd --store https://jokaimor.dyndns.org/owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts
Applying base config from file /home/alendvai/workspace/emacs/ASynK/config/config_v6.json...
Applying base config from file /home/alendvai/workspace/emacs/ASynK/config/config_v6.json...done
Applying user customizations from file /home/alendvai/.asynk/config.py...
Applying user customizations from file /home/alendvai/.asynk/config.py...done
[22:02:50.267     INFO] Debug logging to file: /home/alendvai/.asynk/logs/asynk_logs.2013-12-09.22-02-50.267842
[22:02:50.268     INFO] Deleting log files older than 7 days, if any...
[22:02:50.269     INFO] Deleting log files older than 7 days, if any...done
Please enter your username: attila.lendvai
Password:
[22:02:54.869     INFO] Logging into CardDAV server: https://jokaimor.dyndns.org/owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts - user: attila.lendvai...
Found current principal path: /owncloud/remote.php/carddav/principals/attila.lendvai/
[22:02:59.408     INFO] Logging into CardDAV server: https://jokaimor.dyndns.org/owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts - user: attila.lendvai... done.
[22:02:59.408     INFO] Listing all folders in PIMDB cd...
[22:02:59.409     INFO]   1: Contacts.  Name: contacts; GID: /owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts/;    
[22:02:59.409     INFO] Listing all folders in PIMDB cd...done

this seems to work. but when i try to sync in my bbdb, nothing gets done:

$ ./asynk.py --op=sync --name "owncloud" --sync-all
Applying base config from file /home/alendvai/workspace/emacs/ASynK/config/config_v6.json...
Applying base config from file /home/alendvai/workspace/emacs/ASynK/config/config_v6.json...done
Applying user customizations from file /home/alendvai/.asynk/config.py...
Applying user customizations from file /home/alendvai/.asynk/config.py...done
[22:08:47.234     INFO] Debug logging to file: /home/alendvai/.asynk/logs/asynk_logs.2013-12-09.22-08-47.234663
[22:08:47.235     INFO] Deleting log files older than 7 days, if any...
[22:08:47.236     INFO] Deleting log files older than 7 days, if any...done
[22:08:47.236     INFO] Parsing BBDB file /home/alendvai/.bbdb...
[22:08:47.237     INFO] Parsing BBDB Store with encoding utf-8...
[22:08:47.354     INFO] Parsing BBDB Store with encoding utf-8...Success
[22:08:47.354     INFO] Successfully parsed 352 entries.
[22:08:47.371     INFO] Logging into CardDAV server: https://jokaimor.dyndns.org/owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts/ - user: attila.lendvai...
Found current principal path: /owncloud/remote.php/carddav/principals/attila.lendvai/
[22:08:52.202     INFO] Logging into CardDAV server: https://jokaimor.dyndns.org/owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts/ - user: attila.lendvai... done.
[22:08:52.236     INFO] Deleting BBDB backup files older than 10 days, if any...
[22:08:52.237     INFO] Deleting BBDB backup files older than 10 days, if any...done
[22:08:52.237     INFO] Backedup BBDB Store (/home/alendvai/.bbdb) to file: /home/alendvai/.asynk/backups/bbdb_backup.owncloud.2013-12-09.22-08-52.237287
[22:08:52.238     INFO] Last synk for profile owncloud was at: 1980-01-01T00:00:00.00+00:00
[22:08:54.074     INFO] ==== bb =====
[22:08:54.074     INFO]    New              :     0
[22:08:54.074     INFO]    Modified         :     0
[22:08:54.074     INFO]    Unchanged        :     0
[22:08:54.074     INFO]                       =====
[22:08:54.074     INFO]    Total Entries    :     0
[22:08:54.074     INFO]    Deleted          :     0
[22:08:54.074     INFO] ==== cd =====
[22:08:54.074     INFO]    New              :     0
[22:08:54.074     INFO]    Modified         :     0
[22:08:54.074     INFO]    Unchanged        :     0
[22:08:54.075     INFO]                       =====
[22:08:54.075     INFO]    Total Entries    :     0
[22:08:54.075     INFO]    Deleted          :     0
[22:08:54.075     INFO] Number of entries modified both places (conflicts): 0
[22:08:54.075     INFO] conflict resolve direction : bb. db1id: bb, db2id: cd
[22:08:54.075     INFO] After conflict resolution, size of bb mod :     0
[22:08:54.075     INFO] After conflict resolution, size of cd mod :     0
[22:08:54.075     INFO] After conflict resolution, size of bb del :     0
[22:08:54.075     INFO] After conflict resolution, size of cd del :     0
[22:08:54.075     INFO] =====================================================
[22:08:54.075     INFO]    Sending New bb entries to cd
[22:08:54.076     INFO] =====================================================
[22:08:54.076     INFO] No new entries that need to be synched
[22:08:54.076     INFO] =====================================================
[22:08:54.076     INFO]    Sending Modified bb entries to cd
[22:08:54.076     INFO] =====================================================
[22:08:54.076     INFO] No modified entries that need to be synched
[22:08:54.076     INFO] =====================================================
[22:08:54.076     INFO]    Synching Deleted bb entries to cd
[22:08:54.076     INFO] =====================================================
[22:08:54.076     INFO] No deleted entries that need to be synched.
[22:08:54.076     INFO] =====================================================
[22:08:54.076     INFO]    Sending New cd entries to bb
[22:08:54.076     INFO] =====================================================
[22:08:54.077     INFO] No new entries that need to be synched
[22:08:54.077     INFO] =====================================================
[22:08:54.077     INFO]    Sending Modified cd entries to bb
[22:08:54.077     INFO] =====================================================
[22:08:54.077     INFO] No modified entries that need to be synched
[22:08:54.077     INFO] =====================================================
[22:08:54.077     INFO]    Synching Deleted cd entries to bb
[22:08:54.077     INFO] =====================================================
[22:08:54.077     INFO] No deleted entries that need to be synched.
[22:08:54.111     INFO] Updating item inventory...
[22:08:54.129     INFO] Updating item inventory...done

there's nothing unusual in my config files AFAICS:

      "owncloud" : { "coll_1" : { "dbid" : "bb",
              "foid" : "default",
              "stid" : "~/.bbdb"
            },
          "coll_2" : { "dbid" : "cd",
              "foid" : "/owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts/",
              "stid" : "https://jokaimor.dyndns.org/owncloud/remote.php/carddav/addressbooks/attila.lendvai/contacts/"
            },
          "conflict_resolve" : "bb",
          "items" : {  },
          "last_sync_start" : "2013-12-09T16:08:52.24Z",
          "last_sync_stop" : "2013-12-09T16:08:54.09Z",
          "olgid" : null,
          "sync_dir" : "SYNC2WAY"
        }

Support for birthdays without year

Several of my contacts in Google contacts have birthdays with just the month and day specified (no year). I get the following error for these contacts.

[16:36:17.899 ERROR] birthday for NAME should be yyyy-mm-dd format. Actual value: --01-01

BBBD File processing appears to stop when empty line is encountered

Contact entry cannot be synched to multiple CardDAV servers

Due to an unfortunate decision on assigning itemids to contacts in contact_cd.py items that are synched to one carddav store cannot also be syched to a different server with a different profile. This needs to be fixed. More comments on how this will be addressed in contact_cd.py

KeyError in con_old = cons[href] after merging two bbdb records

I tried using "r" / bbdb-merge-records on two records in BBDB. The sync afterwards failed with this error:

[07:24:50.350 CRITICAL] Exception (u'/owncloud/remote.php/carddav/addressbooks/me/contacts/8cfccf26b5aa556ddf73cc7c54f4a842.vcf, /owncloud/remote.php/carddav/addressbooks/me/contacts/ddbd20f3b370a35bdd056d0759ee14a8.vcf') while syncing profile bbown
[07:24:50.351 CRITICAL] Traceback (most recent call last):
  File "/home/me/src/ASynK/asynk.py", line 786, in op_sync
    result = sync.sync(self.get_sync_dir())
  File "/home/me/src/ASynK/asynk/sync.py", line 298, in sync
    ret1 = sl1.sync_to_folder(self.get_f2())
  File "/home/me/src/ASynK/asynk/sync.py", line 499, in sync_to_folder
    res2 = self.send_mods_to_folder(df)
  File "/home/me/src/ASynK/asynk/sync.py", line 478, in send_mods_to_folder
    res = df.batch_update(self, self.db1id, items)
  File "/home/me/src/ASynK/asynk/folder_cd.py", line 231, in batch_update
    con_old = cons[href]
KeyError: u'/owncloud/remote.php/carddav/addressbooks/me/contacts/8cfccf26b5aa556ddf73cc7c54f4a842.vcf, /owncloud/remote.php/carddav/addressbooks/me/contacts/ddbd20f3b370a35bdd056d0759ee14a8.vcf'

Relevant fields from the record in my ~/.bbdb file:

(bbdb-id . "ddce562a-fd23-11e2-a123-00261898ab57, ddce46c6-fd23-11e2-a123-00261898ab57")
(asynk:bbown:cd . "/owncloud/remote.php/carddav/addressbooks/me/contacts/8cfccf26b5aa556ddf73cc7c54f4a842.vcf, /owncloud/remote.php/carddav/addressbooks/me/contacts/ddbd20f3b370a35bdd056d0759ee14a8.vcf")

I don't know how easy it is to deal with merges, perhaps the documentation should warn that they are to be avoided in the meanwhile …

bbdb <-> cd sync is failing to pick up modified carddav entries

Trying to sync owncloud stable7 with bbdb; if I modify an entry either in the ownCloud webUI or via my Android device (which syncs back to owncloud using DavDroid), the changes are not synced back to bbdb:

[02:07:48.290    DEBUG] Total Contacts   :   252
[02:07:48.291     INFO] ==== bb =====
[02:07:48.291     INFO]    New              :     0
[02:07:48.291     INFO]    Modified         :     0
[02:07:48.291     INFO]    Unchanged        :   226
[02:07:48.292     INFO]                       =====
[02:07:48.292     INFO]    Total Entries    :   226
[02:07:48.292     INFO]    Deleted          :     0
[02:07:48.292     INFO] ==== cd =====
[02:07:48.293     INFO]    New              :     0
[02:07:48.294     INFO]    Modified         :     0
[02:07:48.294     INFO]    Unchanged        :   252
[02:07:48.294     INFO]                       =====
[02:07:48.294     INFO]    Total Entries    :   252
[02:07:48.294     INFO]    Deleted          :     0

To create the profiel I ran asynk --op=create-profile --db bb cd --store ~/.bbdb http://localhost/owncloud/remote.php/carddav/principals/rrix --folder default default --name bbcd

How can I get reliable two-way sync working with oc7 and bbdb? The initial sync works out from bbdb -> cd but it seems like it's one-way despite my state.json having a 2way sync, given the Unchanged numbers in the cd being greater than in bbdb, and the modified entries not being detected:

  "profiles" : { "bbcd" : { "coll_1" : { "dbid" : "bb",
              "foid" : "default",
              "stid" : "/home/rrix/.bbdb"
            },
          "coll_2" : { "dbid" : "cd",
              "foid" : "default",
              "stid" : "http://localhost/owncloud/remote.php/carddav/principals/rrix"
            },
          "conflict_resolve" : "cd",
          "items" : { <TONNES OF IDS> },
          "last_sync_start" : "2014-09-10T09:07:46.98Z",
          "last_sync_stop" : "2014-09-10T09:07:48.34Z",
          "olgid" : null,
          "sync_dir" : "SYNC2WAY",
          "sync_state" : null
        },

Should exit on first 401

$ python2 ./asynk.py --op=sync --user-dir=~/.asynk.carddav --name bbown --dry-run
[16:56:26.021 INFO] Debug logging to file: /home/me/.asynk.carddav/logs/asynk_logs.2013-06-17.16-56-26.021590
Please enter your username:
Password:
[16:56:37.992 INFO] Parsing BBDB file /home/me/.bbdb...
[16:56:37.993 INFO] Parsing BBDB Store with encoding utf-8...
[16:56:38.285 INFO] Parsing BBDB Store with encoding utf-8...Success
[16:56:38.285 INFO] Successfully parsed 519 entries.
401
401
401
401
401
401

… and so on – shouldn't it exit on the first 401?

Deletes need to be synchronized

KeyError: 'nc' with carddav and baikal

Hi,

I'm facing this error message:

$ ./result/bin/asynk.py --op=sync --name carddav2bbdb --dry-run
Applying base config from file ./config/config_v7.json...
Applying base config from file ./config/config_v7.json...done
Applying user customizations from file /home/cassou/.asynk/config.py...
Applying user customizations from file /home/cassou/.asynk/config.py...done
[10:36:22.307     INFO] Debug logging to file: /home/cassou/.asynk/logs/asynk_logs.2015-01-23.10-36-22.306931
[10:36:22.372     INFO] Logging into CardDAV server: https://www.<someserver>.org/baikal/card.php - user: xxx...
Traceback (most recent call last):
  File "asynk.py", line 1126, in <module>
    main()
  File "asynk.py", line 80, in main
    asynk.dispatch()
  File "asynk.py", line 512, in dispatch
    res = getattr(self, self.get_op())()
  File "asynk.py", line 776, in op_sync
    pname = self._load_profile()
  File "asynk.py", line 1120, in _load_profile
    self._login()
  File "asynk.py", line 300, in _login
    self.set_db(db1id, getattr(self, login_func)())
  File "asynk.py", line 1062, in login_cd
    self.get_cduser(), self.get_cdpw())
  File "./asynk/pimdb_cd.py", line 55, in __init__
    self.cd_init()
  File "./asynk/pimdb_cd.py", line 280, in cd_init
    logging=self.get_client_logging())
  File "./lib/caldavclientlibrary/client/account.py", line 23, in __init__
    self.session = CalDAVSession(server, port, ssl, user, pswd, principal, root, logging)
  File "./lib/caldavclientlibrary/client/clientsession.py", line 77, in __init__
    self._initCalDAVState()
  File "./lib/caldavclientlibrary/client/clientsession.py", line 84, in _initCalDAVState
    self._discoverPrincipal()
  File "./lib/caldavclientlibrary/client/clientsession.py", line 89, in _discoverPrincipal
    current = self.getCurrentPrincipalResource(self.rootPath)
  File "./lib/caldavclientlibrary/client/clientsession.py", line 396, in getCurrentPrincipalResource
    hrefs = self.getHrefListProperty(rurl, davxml.current_user_principal)
  File "./lib/caldavclientlibrary/client/clientsession.py", line 267, in getHrefListProperty
    self.runSession(request)
  File "./lib/caldavclientlibrary/client/clientsession.py", line 821, in runSession
    self.doSession(request)
  File "./lib/caldavclientlibrary/client/clientsession.py", line 855, in doSession
    if not self.initialise(self.server, self.rootPath.relativeURL()):
  File "./lib/caldavclientlibrary/protocol/webdav/session.py", line 54, in initialise
    self.sendRequest(request)
  File "./lib/caldavclientlibrary/protocol/http/session.py", line 81, in sendRequest
    self.doRequest(request)
  File "./lib/caldavclientlibrary/client/clientsession.py", line 915, in doRequest
    hdrs = request.getRequestHeaders()
  File "./lib/caldavclientlibrary/protocol/http/requestresponse.py", line 124, in getRequestHeaders
    self.addHeaders(result)
  File "./lib/caldavclientlibrary/protocol/http/requestresponse.py", line 150, in addHeaders
    self.session.addHeaders(hdrs, self)
  File "./lib/caldavclientlibrary/protocol/http/session.py", line 47, in addHeaders
    self.getAuthorization().addHeaders(hdrs, request)
  File "./lib/caldavclientlibrary/protocol/http/authentication/digest.py", line 52, in addHeaders
    os.write(" nc=\"%s\"" % (self.fields['nc'],))
KeyError: 'nc'

I have this profile:

{ [...]
  "profiles" : {
      "carddav2bbdb" : {
          "coll_1" : { "dbid" : "cd",
                       "foid" : "default",
                       "stid" : "https://www.<someserver>.org/baikal/card.php"
                     },
          "coll_2" : { "dbid" : "bb",
                       "foid" : "default",
                       "stid" : "/home/cassou/.bbdb"
                     },
          "conflict_resolve" : "cd",
          "last_sync_start" : "1980-01-01T00:00:00.00+00:00",
          "last_sync_stop" : "1980-01-01T00:00:00.00+00:00",
          "olgid" : null,
          "sync_dir" : "SYNC2WAY",
          "sync_state" : null
      }
  }
}

The caldavclientlibrary is at this version:

$ cd lib/caldavclientlibrary
$ git status
HEAD detached at 06699b0
nothing to commit, working directory clean

The server is a Baikal 0.2.4.

Outlook IMAP/POP accounts need to be timed out

If there are any IMAP or POP accounts in an outlook message store and there is no network connection then application startup takes a lot of time as each account open will time out in turn. There should be a better way to identify and skip such accounts.

docs don't mention need for $ git submodule update --init --recursive

google->bbdb sync fails for new entries: "If-Match or If-None-Match header or entry etag attribute required."

this may be related: https://code.google.com/p/gdata-java-client/issues/detail?id=103

with each sync duplicate entries are accumulating in ~/.bbdb for the new entries.

running the current head: e997225.

the log:

$ ./asynk.py --op=sync --name googletobbdb
Applying base config from file /home/alendvai/workspace/emacs/ASynK/config/config_v7.json...
Applying base config from file /home/alendvai/workspace/emacs/ASynK/config/config_v7.json...done
Applying user customizations from file /home/alendvai/.asynk/config.py...
Applying user customizations from file /home/alendvai/.asynk/config.py...done
[15:38:49.700     INFO] Debug logging to file: /home/alendvai/.asynk/logs/asynk_logs.2014-05-10.15-38-49.700003
[15:38:49.703     INFO] Deleting log files older than 7 days, if any...
[15:38:49.704     INFO] Deleting log files older than 7 days, if any...done
[15:38:49.705     INFO] Logging into Google...
[15:38:52.750     INFO] Parsing BBDB file /home/alendvai/.bbdb...
[15:38:52.750     INFO] Parsing BBDB Store with encoding utf-8...
[15:38:52.893     INFO] Parsing BBDB Store with encoding utf-8...Success
[15:38:52.894     INFO] Successfully parsed 372 entries.
[15:38:52.894     INFO] Deleting BBDB backup files older than 10 days, if any...
[15:38:52.897     INFO] Deleting BBDB backup files older than 10 days, if any...done
[15:38:52.898     INFO] Backedup BBDB Store (/home/alendvai/.bbdb) to file: /home/alendvai/.asynk/backups/bbdb_backup.googletobbdb.2014-05-10.15-38-52.897990
[15:38:52.899     INFO] Last synk for profile googletobbdb was at: 2014-03-20T07:37:56.84Z
[15:39:25.405     INFO] Querying Google for status of Contact Entries...
[15:39:27.537     INFO] Response recieved from Google. Processing...
[15:39:27.545     INFO] ==== gc =====
[15:39:27.546     INFO]    New              :     4
[15:39:27.546     INFO]    Modified         :    21
[15:39:27.546     INFO]    Unchanged        :   330
[15:39:27.547     INFO]                       =====
[15:39:27.547     INFO]    Total Entries    :   355
[15:39:27.547     INFO]    Deleted          :     0
[15:39:27.548     INFO] =====================================================
[15:39:27.548     INFO]    Sending New gc entries to bb
[15:39:27.548     INFO] =====================================================
[15:39:27.548     INFO] 4 new entries to be synched.
[15:39:29.469     INFO] Successfully created BBDB entry for           name1 (fbb7055e-)
[15:39:29.477     INFO] Successfully created BBDB entry for                  name2 (fbb835c8-)
[15:39:29.485     INFO] Successfully created BBDB entry for               name 3 (fbb97cb2-)
[15:39:29.493     INFO] Successfully created BBDB entry for                  name 4 (fbbab6a4-)
[15:39:29.493     INFO] Saving BBDB File /home/alendvai/.bbdb...
[15:39:29.549     INFO] Saving BBDB File /home/alendvai/.bbdb...done
[15:39:29.550     INFO] Uploading Remote ItemIDs to Google...
[15:39:31.038    ERROR] Could not complete sync for: fbb7055e-: Reason: If-Match or If-None-Match header or entry etag attribute required. Code: SYNC_FORBIDDEN: <ns0:id xmlns:ns0="http://www.w3.org/2005/Atom">https://www.google.com/m8/feeds/contacts/...</ns0:id>
[15:39:31.038    ERROR] Could not complete sync for: fbb835c8-: Reason: If-Match or If-None-Match header or entry etag attribute required. Code: SYNC_FORBIDDEN: <ns0:id xmlns:ns0="http://www.w3.org/2005/Atom">https://www.google.com/m8/feeds/contacts/...</ns0:id>
[15:39:31.038    ERROR] Could not complete sync for: fbb97cb2-: Reason: If-Match or If-None-Match header or entry etag attribute required. Code: SYNC_FORBIDDEN: <ns0:id xmlns:ns0="http://www.w3.org/2005/Atom">https://www.google.com/m8/feeds/contacts/...</ns0:id>
[15:39:31.039    ERROR] Could not complete sync for: fbbab6a4-: Reason: If-Match or If-None-Match header or entry etag attribute required. Code: SYNC_FORBIDDEN: <ns0:id xmlns:ns0="http://www.w3.org/2005/Atom">https://www.google.com/m8/feeds/contacts/...</ns0:id>
[15:39:31.039     INFO] =====================================================
[15:39:31.039     INFO]    Sending Modified gc entries to bb
[15:39:31.039     INFO] =====================================================
[15:39:31.039     INFO] 21 modified entries to be synched.
[15:39:34.386     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.387     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.388     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.389     INFO] Successfully updated BBDB entry for     ... (f8...)
[15:39:34.389     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.390     INFO] Successfully updated BBDB entry for     ... (f8...)
[15:39:34.391     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.392     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.392     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.393     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.394     INFO] Successfully updated BBDB entry for     ... (f8...)
[15:39:34.395     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.395     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.396     INFO] Successfully updated BBDB entry for     ... (b3...)
[15:39:34.397     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.398     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.399     INFO] Successfully updated BBDB entry for     ... (b3...)
[15:39:34.400     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.401     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.401     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.402     INFO] Successfully updated BBDB entry for     ... (be...)
[15:39:34.402     INFO] Saving BBDB File /home/alendvai/.bbdb...
[15:39:34.447     INFO] Saving BBDB File /home/alendvai/.bbdb...done
[15:39:34.448     INFO] =====================================================
[15:39:34.448     INFO]    Synching Deleted gc entries to bb
[15:39:34.448     INFO] =====================================================
[15:39:34.448     INFO] No deleted entries that need to be synched.
[15:39:34.448     INFO] timestamps not reset for profile googletobbdb due to errors (previously identified).

Fatal error when syncing gc and bb

I am seeing the following error when syncing gc and bb.

[21:25:22.707 ERROR] Unknown fatal error in response. Full resp: <ns0:entry xmlns:ns0="http://www.w3.org/2005/Atom" xmlns:ns1="http://schemas.google.com/gdata/batch">ns0:idhttps://www.google.com/m8/feeds/contacts/admin%40nordrandklinik.at/full/batch/1388953521169/fatal/ns0:idns0:updated2014-01-05T20:25:21.178Z/ns0:updatedns0:contentFeed processing was interrupted./ns0:contentns0:titleFatal Error/ns0:title<ns1:interrupted error="0" parsed="74" reason="[Line 365, Column 1893, element ns2:userDefinedField] Missing attribute: 'key'" success="0" unprocessed="74" />/ns0:entry

I have many contacts ( > 2000) and therefore tried syncing with a small subset of them. In this case there was no error. I suspect that there might be contacts with offending entries, However I am unable to locate them.

I am using the latest async.py from Git and GNU Emacs on Win7.

[Feature request] support CardDAV

Hi,
It would be great if ASynk would support CardDAV so that people could use whichever host they desired and not just Google. In particular bbdb to CardDAV would be cool. . .

Could not parse BBDB record

I'm using BBDB 3, but my ~/.bbdb was converted from BBDB 2. I seem to get "Could not parse BBDB record" for most records in my file. Here's one of them:

["promo telecom" "2A" nil nil ("ENSIMAG") nil nil ("[email protected]") ((creation-date . "2002-01-07") (timestamp . "2002-01-07") (mail-alias . "telecom2a")) nil]

Sync between two Google Contacts accounts

When I try to set up a profile to sync from [email protected] and [email protected], ASynK misunderstands and creates a profile to sync [email protected] to [email protected].

Even if I manually change state.json to fix the profile, I don't think I can provide the password either with .netrc (the naming doesn't seem to support providing the password for two accounts using the same provider) or the command-line.

Could you support GC-GC sync?

Sync Outlook Tasks to Google Calendar

Google Tasks is a piece of crap. But Google Calendar is a pretty awesome product. It appears like most of the Outlook Task properties can be mapped to Google Calendar entries, to make a bi-directional sync possible. This needs some serious consideration.

BBDB field values can be arbitrary lisp expressions

As of this BBDB commit, it's possible to use general lisp expressions as the value of BBDB contact fields. So a plain old list, or a struct, or what have you.

Understandably, this breaks ASynK's parsing of the record -- I don't expect there'd be any way of reliably parsing that stuff in Python.

Is there any way to gracefully fail in these situations, and simply omit the offending fields? Also, I checked the documentation, and there doesn't seem to be any way to specify BBDB fields to ignore, is that right? Of course, any ignoring would probably happen after parsing, so maybe that wouldn't help anyway...

Thanks for any suggestions.

Need a proper Unit Test Suite

There is currently a test/ directory in the application root, but not all work any more, and not all are mean to be executed as a part of any automated pre-release certification test. This is not a great state of affairs. We need a proper suite of tests to ensure all the fields are synched properly across all pairs of supported dbs.

But this is a LOOOOT of work... Hm.

AttributeError: 'NoneType' object has no attribute 'authenticators'

When I try to run an list_folders or sync operation, I get an error. Running asynk.py, asynk.py -h, or asynk.py -op=create-profile … does not produce any errors.

My python version is 2.7.9. Cleaning the configuration and running

python2.7 asynk.py --op=sync --dry-run

produces:

Traceback (most recent call last):
File "asynk.py", line 302, in <module>
    main()
File "asynk.py", line 299, in main
    asynk.dispatch()
File "/home/user/Gits/ASynK/asynk/asynk_core.py", line 97, in dispatch
    res = getattr(self, self.get_op())()
File "/home/user/Gits/ASynK/asynk/asynk_core.py", line 116, in op_list_folders
    self._login()
File "/home/user/Gits/ASynK/asynk/asynk_core.py", line 84, in _login
    coll.init_username_pwd()
File "/home/user/Gits/ASynK/asynk/state_collection.py", line 214, in init_username_pwd
    self.get_colln())
File "/home/user/Gits/ASynK/asynk/state_collection.py", line 65, in get_auth
    res = self.netrc.authenticators(mach)
AttributeError: 'NoneType' object has no attribute 'authenticators'

Do you have any idea what might be causing this? Are there any addiotional tests I could perform?

Old CalDAV library

AsynK uses CalDAVClientLibrary which seems unmaintained. It looks like there are alternative ones: http://caldav.calconnect.org/implementations/librariestools.html

--name not set by create-profile?

Tried running ASynK on Arch Linux with my owncloud:

$ python2 ./asynk.py --op=create-profile --name bbown --user-dir=~/.asynk.carddav --db bb cd --folder default default --store ~/.bbdb 'https://me.net:443/owncloud/remote.php
/carddav/addressbooks/me/contacts'
Creating ASynK User directory at: /home/me/.asynk.carddav
[23:02:26.102 INFO] Creating Logs directory at: /home/me/.asynk.carddav/logs
[23:02:26.103 INFO] Debug logging to file: /home/me/.asynk.carddav/logs/asynk_logs.2013-06-16.23-02-26.103537
[23:02:26.103 INFO] Deleting log files older than 7 days, if any...
[23:02:26.104 INFO] Deleting log files older than 7 days, if any...done
[23:02:26.104 INFO] Successfully added profile: bbown

seems OK, but then:

$ python2 ./asynk.py --op=sync --user-dir=~/.asynk.carddav --name bbown --dry-run
[23:17:16.756 INFO] Debug logging to file: /home/me/.asynk.carddav/logs/asynk_logs.2013-06-16.23-17-16.756198
Traceback (most recent call last):
File "./asynk.py", line 1020, in
main()
File "./asynk.py", line 81, in main
asynk.dispatch()
File "./asynk.py", line 476, in dispatch
res = getattr(self, self.get_op())()
File "./asynk.py", line 693, in op_sync
pname = self._load_profile()
File "./asynk.py", line 1001, in _load_profile
self.set_db1(conf.get_profile_db1(pname))
File "/home/me/src/ASynK/asynk/state.py", line 389, in get_profile_db1
return self.get_coll_1(profile)['dbid']
File "/home/me/src/ASynK/asynk/state.py", line 377, in get_coll_1
return self._get_profile_prop(profile, 'coll_1')
File "/home/me/src/ASynK/asynk/state.py", line 247, in _get_profile_prop
% profile)
state.AsynkConfigError: Profile bbown not found in state.json

and sure enough,

$ grep bbown ~/.asynk.carddav/state.json

gives nothing. So I try removing --name bbown to see if it's just that
the name wasn't set correctly by --op=create-profile:

$ python2 ./asynk.py --op=sync --user-dir=~/.asynk.carddav --dry-run
[23:18:57.136 INFO] Debug logging to file: /home/me/.asynk.carddav/logs/asynk_logs.2013-06-16.23-18-57.136457
Please enter your username:
Password:
[23:19:28.662 INFO] Logging into Google...
Traceback (most recent call last):
File "./asynk.py", line 1020, in
main()
File "./asynk.py", line 81, in main
asynk.dispatch()
File "./asynk.py", line 476, in dispatch
res = getattr(self, self.get_op())()
File "./asynk.py", line 693, in op_sync
pname = self._load_profile()
File "./asynk.py", line 1014, in _load_profile
self._login()
File "./asynk.py", line 326, in _login
self.set_db(db1id, getattr(self, login_func)())
File "./asynk.py", line 953, in login_gc
raise AsynkError('Invalid Google credentials. Cannot proceed.')
main.AsynkError: Invalid Google credentials. Cannot proceed.

Did I just send my username and password to Google??

Neighbourhood and PO Box info from GC gets lost in round trip

These two fields if explicitly structured at GC end can get lost. These need to be handled explicitly.

Handle non-ascii chars in bbdb

bbdb / emacs has a rather wacky way of representing non-ASCII characters.

ASynck, encountering them, throws errors on the command line like:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 53: ordinal not in range(128)

The debugger shows that python is calling str(e), where e is:

BBDBParseError(u'Could not Parse BBDB contact entry: ["D Jared" #("Dom\xednguez" 0 9 (charset iso-8859-1)) nil nil nil nil ("[email protected]") ((creation-date . "2011-10-31") (timestamp . "2011-10-31")) nil]',)

Here are some more bbdb lines that produce the same error:
[#("Héctor" 0 6 (charset iso-8859-1)) "Tarrido-Picart" nil nil nil nil ("[email protected]") ((creation-date . "2011-12-14") (timestamp . "2011-12-14")) nil]
[#("severiano alberto vélez" 0 23 (charset iso-8859-1)) #("gómez" 0 5 (charset iso-8859-1)) nil nil nil nil ("[email protected]") ((creation-date . "2011-12-14") (timestamp . "2011-12-14")) nil]
[#("王洋" 0 2 (charset chinese-gb2312)) "" nil nil nil nil ("[email protected]") ((creation-date . "2011-10-06") (timestamp . "2011-10-06")) nil]
[#("박준원" 0 3 (charset korean-ksc5601)) "" nil nil nil nil ("[email protected]") ((creation-date . "2011-10-06") (timestamp . "2011-10-06")) nil]

If this issue is still open in a month, I'll dig deeper, but I don't have time now.

crash on non-ascii in ~/.netrc

even though I don't use .netrc for asynk, it crashes on reading non-ascii characters in that file.

Outlook sync fails without error when Outlook is not running

Not yet been able to reproduce, but essentially there is a problem, but this morning, I restarted my VMWare fusion windows instance, and ran a sync. It went through without any issues, but no changes were saved to Outlook when I opened to check. In outlook it appeared like nothing had happened.

Ran the sync again (after rolling back the BBDB with much pain), and with Outlook open this time, the sync really did work. Log file is available as asynk_logs.2012-07-02.08-04-27.862000 on the dev machine

ImportError: cannot import name vobject

asynk.py now fails with this message on my machine. I'm running Ubuntu 12.10 and installed python-vobject.

Better password protection

The password field currently displays the password in clear text. I see a couple mentions of getpass, but it seems there is more than one code path to enter a password. A better system is to support running a process to return the password, as in gpg -d mypass.gpg, or even just .netrc.

CardDAV server 500 need to be handled properly.

CalDAVClientLibrary seems to not be raising proper exceptions, causing problems.

do first default sync with Google, delete .bbdb file, resync deletes all Google contacts

Hi, experimenting, I decided to remove .bbdb after my first test sync to see what would happen.

A subsequent sync deleted all contacts in Google, as deleted .bbdb is "newer".

Exception during sync results in incomplete BBDB FIle

If there is a syntax error in the code or whatever, and the sync terminates, the BBDB database is truncated at that point. Nothing more gets written. This is unacceptable. The best possible case is the backup should be automatically copied over to the store so the user does not find anything amiss.

Detailed sync statistics for a given folder

A folder can be a part of any number of sync profiles. We should have a simple query operation to find out the statistics of how many are synced, how many are new, how many updated since last sync, etc. for each profile.

Preamble:

1. Folder Name
2. Folder ID
3. Number of Items in folder at this time

Then we will need to print a table with following columns:

1. Profile Name
2. Number of items newly created since last sync for this profile
3. Number of items modified since last sync for this profile
4. Number of items deleted since last sync for this profile