offlineimap/offlineimap

Read/sync your IMAP mailboxes (python2) [LEGACY: move to offlineimap3]

Financial contributors:

Links:

• Official github code repository: offlineimap
• Website: website
• Wiki: wiki
• Blog: blog

OfflineIMAP

"Get the emails where you need them."

IMPORTANT NOTE: This repository is for python2 only. The support for offlineimap3 is happening in Official offlineimap for python3.

I'll still lazily maintain this legacy offlineimap but users should definitely go with offlineimap3.

• Official offlineimap for python3.
• Official offlineimap for python2.

Description

OfflineIMAP is software that downloads your email mailbox(es) as local Maildirs. OfflineIMAP will synchronize both sides via IMAP.

Why should I use OfflineIMAP?

IMAP's main downside is that you have to trust your email provider to not lose your email. While certainly unlikely, it's not impossible. With OfflineIMAP, you can download your Mailboxes and make you own backups of your Maildir.

This allows reading your email offline without the need for your mail reader (MUA) to support IMAP operations. Need an attachment from a message without internet connection? No problem, the message is still there.

Project status and future

The offlineimap project was forked to offlineimap3 to support python3. Contributions are welcome to this project.

Contributors

Code Contributors

This project exists thanks to all the people who contribute. [Contribute].

Financial Contributors

Become a financial contributor and help us sustain our community. [Contribute]

Individuals

Organizations

Support this project with your organization. Your logo will show up here with a link to your website. [Contribute]

License

GNU General Public License v2.

Downloads

You should first check if your distribution already packages OfflineIMAP for you. Downloads releases as tarball or zipball.

If you are running Linux Os, you can install offlineimap with:

• openSUSE zypper in offlineimap
• Arch Linux pacman -S offlineimap
• fedora dnf install offlineimap

Feedbacks and contributions

The user discussions, development, announcements and all the exciting stuff take place on the mailing list. While not mandatory to send emails, you can subscribe here.

Bugs, issues and contributions can be requested to both the mailing list or the official Github project. Provide the following information:

• system/distribution (with version)
• offlineimap version (offlineimap -V)
• Python version
• server name or domain
• CLI options
• Configuration file (offlineimaprc)
• pythonfile (if any)
• Logs, error
• Steps to reproduce the error

The community

• OfflineIMAP's main site is the project page at Github.
• There is the OfflineIMAP community's website.
• And finally, the wiki.

Requirements & dependencies

• Python v2.7.x

• six (required)

• rfc6555 (required)

• imaplib2 >= 2.57 (optional)

• gssapi (optional), for Kerberos authentication

• portalocker (optional), if you need to run offlineimap in Cygwin for Windows

• Python v3: See the offlineimap3 fork of offlineimap.

Documentation

All current and updated documentation is on the community's website.

Read documentation locally

You might want to read the documentation locally. Get the sources of the website. For the other documentation, run the appropriate make target:

$ ./scripts/get-repository.sh website
$ cd docs
$ make html  # Requires rst2html
$ make man   # Requires a2x (http://asciidoc.org)
$ make api   # Requires sphinx

offlineimap's Issues

apply foldersort after folderfilter

Hi,
the 'foldersort' option doesn't work, it's completely ignored. Also there's no mention of it in man page.

Default to distro default ca-certicates file

Hi,

when doing the SSL verification it would be nice if by default offlineimap would use the distro ca-certificates file to verify against. For ubuntu this would be /etc/ssl/certs/ca-certificates.crt (fedora seems to use /etc/pki/tls/certs/ca-bundle.crt suse /etc/ssl/ca-bundle.pem).

If you agree with this idea I can provide a small patch for this.

Cheers,
Michael

P.S. https://bugs.launchpad.net/ubuntu/+source/offlineimap/+bug/1015692 is the ubuntu bugreport for this.

sync all but deletions on local or remote

Could you add new option for local repositories? Something like saveonly = True|False.
Default would be False (current behavior).When True no already saved message nor created folder should be deleted from local repository even if deleted from remote one.

This change would make OfflineIMAP perfect for IMAP account archival (combined with readonly = True on remote repository).

Would you consider it?

Suspend computer with IDLE -> no emails fetched

Hi,

I'm running OfflineIMAP IDLE-mode as I want fast mail notification but when I put my computer to sleep and wake it up the process seems to seems to hang as it does not fetch any new mails. If I run it in a shell and observe stdout after the wakeup it does simply not say anything at all. If I strace it it just keeps printing

select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)

The exact configuration I'm using can be seen in my dotfiles repo and the version I'm using in 6.5.4.

I'm not pleases with having to kill and restart OfflineIMAP each time I sleep my computer :) Any ways of fixing this?

Several mbnames sections

Hi,

I would like to generate several files via different mbnames sections. I've tried putting 2 mbnames sections (one to generate the mutt mailboxes, the other to generate my .checkmailrc); but only the last one gets interpreted.
If there is a possibility to upstream, I may write a patch.

offlineimap is not terminated properly

Hi,

I'm using offlineimap in Debian SID, after run offlineimap, all input will not be echoed, and the command output endline becomes fuzzy: this is a simple output:

liang@guoliangc:/z/src/offlineimap/offlineimap(master) $ akonadi-liang.X20Cgy gpg-LSpLDX kde-liangYH651W liang offlineimap_6.5.5-0.1.diff.9cydkl orbit-liang
a.svg hsperfdata_liang ksocket-kdm mp.log oi.log ssh-kppxVIMeUtcQ
fcitx-socket-:0 kde-kdm ksocket-liang offlineimap ol.log
liang@guoliangc:/z/src/offlineimap/offlineimap(master) $

After some dig, I found tty properties have been chaned, with a normal console, the stty shows:
speed 38400 baud; line = 0;
-brkint ixoff -imaxbel iutf8

with offlineimap runned console, the stty shows:
speed 38400 baud; line = 0;
-brkint ixoff -imaxbel iutf8
-onlcr
-echo
-onlcr and -echo are added.

-onlcr and -echo is seted in init_banner() and should be removed in terminate(), but when exit normally, terminate() is not called.

Thanks,

RC on PyPI

The only release of offlineimap available on the Python Package Index (http://pypi.python.org) is a release candidate.

This renders the package non-installable without using --pre to allow pre-releases.

hh@mail:~$ pip install --user offlineimap
Downloading/unpacking offlineimap
  Could not find a version that satisfies the requirement offlineimap (from versions: 6.5.5-rc2)
Cleaning up...
No distributions matching the version for offlineimap
Storing complete log in /home/hh/.pip/pip.log
hh@mail:~$ cat .pip/pip.log 
Downloading/unpacking offlineimap

  Getting page https://pypi.python.org/simple/offlineimap/
  URLs to search for versions for offlineimap:
  * https://pypi.python.org/simple/offlineimap/
  Analyzing links from page https://pypi.python.org/simple/offlineimap/
    Found link https://pypi.python.org/packages/source/o/offlineimap/offlineimap-6.5.5-rc2.tar.gz#md5=cdef488bc0c188a6cf15bf7dd060abd1 (from https://pypi.python.org/simple/offlineimap/), version: 6.5.5-rc2
  Ignoring link https://pypi.python.org/packages/source/o/offlineimap/offlineimap-6.5.5-rc2.tar.gz#md5=cdef488bc0c188a6cf15bf7dd060abd1 (from https://pypi.python.org/simple/offlineimap/), version 6.5.5-rc2 is a pre-release (use --pre to allow).
  Could not find a version that satisfies the requirement offlineimap (from versions: 6.5.5-rc2)

Cleaning up...

  Removing temporary dir /tmp/pip_build_hh...
No distributions matching the version for offlineimap

Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1085, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/index.py", line 309, in find_requirement
    raise DistributionNotFound('No distributions matching the version for %s' % req)
DistributionNotFound: No distributions matching the version for offlineimap

IDLE working only once per sync

With the idlefolders setting on, IDLE only retrieves one new message in between full syncs. Subsequent messages are not retrieved by IDLE until the next full sync.

I'm using offlineimap 6.5.5 on arch linux, installed from the official repositories (https://www.archlinux.org/packages/community/any/offlineimap/) and connecting to a corporate exchange server.

EDIT: After toying around with the offlineimap sources, it appears that the issue lies in the acquireconnection() method of imapserver.py. I am able to successfully receive multiple IDLE updates if I create the connection directly in the IdleThread class using offlineimap.imaplib2.IMAP4_SSL(servername) and logging in via imapobj.login(user, pass). I'm going to take a bit more in depth look at the acquireconnection() code tomorrow.

with -a CLI option, mbnames must not erase the entries of other accounts

The current behaviour is that mbnames are constructed when an account is synchronized. When using the -a command line option to override the accounts from config file, mbnames are constructed only for the accounts specified on command line. It would be much more useful if the -a option caused that mbnames were constructed for all accounts from the config file.

Intended setup: I want to have imapfilter running in an infinite loop, which would then run offlineimap only for specific account when a new mail is delivered. Currently it is needed to synchronize all accounts to have correct mbnames.

ERROR: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

Hello,
I upgraded from offlineimap 6.5.4 to 6.5.5 a few minutes ago, and it appears I cannot fetch my emails anymore.

The full traceback is:

Traceback:
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 241, in syncrunner
    self.sync()
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 303, in sync
    remoterepos.getfolders()
  File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 313, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 415, in acquireconnection
    self._authn_helper(imapobj)
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 325, in _authn_helper
    if func(imapobj):
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 251, in _authn_plain
    imapobj.authenticate('PLAIN', self.plainhandler)
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplib2.py", line 648, in authenticate
    typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplib2.py", line 1615, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplib2.py", line 1354, in _command
    literal = literator(data, rqb)
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplib2.py", line 2163, in process
    ret = self.mech(self.decode(data))
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 166, in plainhandler
    retval = NULL.join((authz, authc, passwd)).encode('utf-8')

This is due to having an unicode character in my passphrase; I changed “passwd” to “passwd.decode('utf-8')” in the line above and it works fine. Of course, this is only a quick hack in order to access my emails.

ERROR: IMAP server 'Server Remote' does not have a message with UID 'xxx'

There are a number of emails in my mailbox that constantly raise the exception below

raise OfflineImapError(reason, severity)

The errors are something like

ERROR: IMAP server 'Server ### Remote' does not have a message with UID 'xxx'

Any idea on what's going on?

This is with version 6.5.5

enable packaging with PyPi

Happens on Ubuntu and OS X.

$ pip install offlineimap
Downloading/unpacking offlineimap
Downloading offlineimap-6.5.5-rc2.tar.gz (97kB): 97kB downloaded
Running setup.py egg_info for package offlineimap
Traceback (most recent call last):
File "", line 16, in
File "/home/ahknight/.virtualenvs/imap/build/offlineimap/setup.py", line 28, in
from test.OLItest import TextTestRunner, TestLoader, OLITestLib
ImportError: No module named OLItest
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 16, in

File "/home/ahknight/.virtualenvs/imap/build/offlineimap/setup.py", line 28, in

from test.OLItest import TextTestRunner, TestLoader, OLITestLib

ImportError: No module named OLItest

Deadlock in OfflineImap

I'm experiencing issues with OfflineImap upstream, basically it deadlocks.
I'm running OI in a single thread mode (-1) for an account with 1000's of mails to sync.
It locks after 102 downloaded messages (see below for an explaination of this number)

I've find out the thread stack traces while it's struck:

# ThreadID: 1133593712
File: "/usr/lib/python2.7/threading.py", line 524, in __bootstrap
  self.__bootstrap_inner()
File: "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
  self.run()
File: "/home/cyril/offlineimap/offlineimap/threadutil.py", line 227, in run
  ExitNotifyThread.run(self)
File: "/home/cyril/offlineimap/offlineimap/threadutil.py", line 175, in run
  exitthreads.put(self, True)
File: "/usr/lib/python2.7/Queue.py", line 126, in put
  self.not_full.wait()
File: "/usr/lib/python2.7/threading.py", line 243, in wait
  waiter.acquire()




# ThreadID: 1125205104
File: "/usr/lib/python2.7/threading.py", line 524, in __bootstrap
  self.__bootstrap_inner()
File: "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
  self.run()
File: "/home/cyril/offlineimap/offlineimap/threadutil.py", line 227, in run
  ExitNotifyThread.run(self)
File: "/home/cyril/offlineimap/offlineimap/threadutil.py", line 156, in run
  Thread.run(self)
File: "/usr/lib/python2.7/threading.py", line 504, in run
  self.__target(*self.__args, **self.__kwargs)
File: "/home/cyril/offlineimap/offlineimap/accounts.py", line 446, in syncfolder
  remotefolder.syncmessagesto(localfolder, statusfolder)
File: "/home/cyril/offlineimap/offlineimap/folder/Base.py", line 534, in syncmessagesto
  action(dstfolder, statusfolder)
File: "/home/cyril/offlineimap/offlineimap/folder/Base.py", line 413, in syncmessagesto_copy
  thread.start()
File: "/home/cyril/offlineimap/offlineimap/threadutil.py", line 222, in start
  instancelimitedsems[self.instancename].acquire()
File: "/usr/lib/python2.7/threading.py", line 323, in acquire
  self.__cond.wait()
File: "/usr/lib/python2.7/threading.py", line 243, in wait
  waiter.acquire()




# ThreadID: 1074061904
File: "offlineimapTraced.py", line 26, in <module>
  oi.run()
File: "/home/cyril/offlineimap/offlineimap/init.py", line 48, in run
  self.sync(options)
File: "/home/cyril/offlineimap/offlineimap/init.py", line 356, in sync
  self.sync_singlethreaded(syncaccounts)
File: "/home/cyril/offlineimap/offlineimap/init.py", line 381, in sync_singlethreaded
  account.syncrunner()
File: "/home/cyril/offlineimap/offlineimap/accounts.py", line 240, in syncrunner
  self.sync()
File: "/home/cyril/offlineimap/offlineimap/accounts.py", line 329, in sync
  thread.start()
File: "/home/cyril/offlineimap/offlineimap/threadutil.py", line 222, in start
  instancelimitedsems[self.instancename].acquire()
File: "/usr/lib/python2.7/threading.py", line 323, in acquire
  self.__cond.wait()
File: "/usr/lib/python2.7/threading.py", line 243, in wait
  waiter.acquire()

Basically, for each message that's downloaded from the IMAP server, the code goes in
syncmessagesto_copy which triggers a new thread (this is a bug in a "-1/single-thread" launch already, but still, it should not deadlock) for copying the message to the local folder.
This deadlocks because, this (new) thread is an InstanceLimitedThread which is, in turns, is a ExitNotifyThread, and the threadInstanceLimits is set to 1 in single thread mode (which, I think it correct).

Because ExitNotifyThread is using a size-limited queue to store the exit messages/exceptions/state, this blocks as soon as you've filled the complete Queue.
The Queue is can contains 100 items, so it blocks after after 101 msg (exactly one more, since I've run in -1 mode).

To avoid this deadlock, some thread should empty/pop the ExitNotifyThread queue, so the queue never fills up.

In addition, to fix the other bug, the IMAPFolder should not return true on suggestthreads() if launched with -1 argument.

non-existing mailbox on server makes account stop syncng

currently, after getting this error on one of the configured accounts, no more sync is done for the failed account. this could maybe just log an error and continue gracefully?

 Hook return code: 1
 *** Finished account 'testaccount' in 0:37
 Next refresh in 5.0 minutes
Exception in thread Thread-14:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/dist-packages/offlineimap/imapserver.py", line 545, in idle
    raise e
OfflineImapError: Error SELECTing mailbox 'INBOX.Alerts', server reply:
('NO', ["Mailbox doesn't exist: INBOX.Alerts"])

Add how to specify password without using manual input to FAQ

The documentation touts "There are six different ways to hand the IMAP password to OfflineImap from console input, specifying in the configuration file, .netrc support, specifying in a separate file, to using arbitrary python functions that somehow return the password."

I tried to find this but was unable to find all the different ways actually explained.

I want to be able to use the OSX keychain to do this like getmail does it. ( security find-internet-password -w -a '[email protected]' for example ).

If the six different ways were described, I could perhaps found something that works for me, but I could not locate these in the docs. Perhaps since this is on the front-page of the website, there could be a link describing these six ways in detail

mbnames: localfolder

in mbnames you can specify a peritem string, which expands some tokens: accountname and foldername.
Unfortunately, this is not enough for mutt, as the accountname could be different from the localfolders" specified in the localrepository

This should be quite easy to do, just some lines in mbnames.py

AssertionError: Copy message -1

Syncing lavoro: IMAP -> Maildir
Copy message -1 Maildir[lavoro] -> IMAP[lavoro]
WARNING: ERROR attempting to copy message -1 for account riseup:Traceback (most recent call last):
 File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 255, in copymessageto
 newuid = dstfolder.savemessage(uid, message, flags, rtime)
 File "/usr/lib/pymodules/python2.7/offlineimap/folder/IMAP.py", line 460, in savemessage
 assert(typ == 'OK')
AssertionError

WARNING: ERROR attempting to sync flags for account riseup:Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 421, in syncmessagesto
action(dstfolder, statusfolder)
File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 318, in syncmessagesto_copy
self.copymessageto(uid, dstfolder, statusfolder, register = 0)
File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 255, in copymessageto
newuid = dstfolder.savemessage(uid, message, flags, rtime)
File "/usr/lib/pymodules/python2.7/offlineimap/folder/IMAP.py", line 460, in savemessage
assert(typ == 'OK')
AssertionError

WARNING: ERROR in syncfolder for riseup folder lavoro: Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/offlineimap/accounts.py", line 355, in syncfolder
localfolder.syncmessagesto(remotefolder, statusfolder)
File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 421, in syncmessagesto
action(dstfolder, statusfolder)
File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 318, in syncmessagesto_copy
self.copymessageto(uid, dstfolder, statusfolder, register = 0)
File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 255, in copymessageto
newuid = dstfolder.savemessage(uid, message, flags, rtime)
File "/usr/lib/pymodules/python2.7/offlineimap/folder/IMAP.py", line 460, in savemessage
assert(typ == 'OK')
AssertionError

This is what Offlineimap 6.3.4 on Ubuntu 12.04 has issued today when syncing one of my 3 mail accounts from online to local storage.

Any hint?

Somehow v6.5.4 deletes the entire INBOX

With a misconfiguration, OfflineIMAP synchronized an empty local INBOX folder to a remote (full) INBOX, and deleted everything in the remote INBOX.

After restoring the remote INBOX from backups, and finishing a full re-sync, OfflineIMAP nuked the remote and local INBOXes again :(

Folder INBOX [acc: Personal]:
 Copy message 16931 (5 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16932 (6 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16933 (7 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16934 (8 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16935 (9 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16936 (10 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16937 (11 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16938 (12 of 13) Local_Personal:INBOX -> Remote_Personal
 Copy message 16939 (13 of 13) Local_Personal:INBOX -> Remote_Personal
 Deleting 16885 messages (213268:213270,213272:213274,213277:213279,213281:214753,214756:229375,230093:230657,230659:230875,230879) in IMAP[INBOX]

I don't know what information to add, nothing looks erroneous, it just looks like OfflineIMAP tried to correctly delete the messages I deleted locally "(213268:213270,213272:213274,213277:213279,213281:214753,214756:229375,230093:230657,230659:230875,230879)" but instead decided to delete everything instead.

:(

'NoneType' object has no attribute 'group'

Upon upgrading to 6.5.5 RC2, I'm still getting the following error when syncing mail:

ERROR: Exceptions occurred during the run! ERROR: While attempting to sync account 'Personal' 'NoneType' object has no attribute 'group'

The mail seems to be syncing fine though as I getting my mail. If there's any more output I can provide to further diagnose this, please let me know.

Non ASCII in folder names cannot sync

My offlineimap installation worked perfectly with my Gmail until I upgraded my Ubuntu to 12.10. Now offlineimap fails with errors like:

'ascii' codec can't decode byte 0xe7 in position 1: ordinal not in range(128)

I tried with both the Ubuntu version and git version of offlineimap.

OfflineIMAP does not follow XDG Base Spec

OfflineIMAP stores its configuration in ~/.offlineimaprc. Instead of cluttering the home directory, it should follow the XDG Base Directory spec and store its configuration in its own directory under $XDG_CONFIG_HOME (.config by default).

http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

autorefresh blocks other accounts from syncing.

If you have three accounts set up to sync, if the first is autorefreshing, then it blocks the other accounts from refreshing.

INBOX is not synced with Exchange server behind davmail

The Exchange server in my company has no IMAP.
So I use davmail (port 1143) to work as a bridge and sync the exchange email to a local dovecot (port 143).

However all folders are synced well except the INBOX.
The output of ttyui is

shiquanwang@las-z820:~$ offlineimap -a Davmail
OfflineIMAP 6.5.4
  Licensed under the GNU GPL v2+ (v2 or any later version)
Account sync Davmail:
 *** Processing account Davmail
 Establishing connection to localhost:1143
 Establishing connection to localhost:143
 Creating folder INBOX/test[LocalDav]
Folder INBOX/test [acc: Davmail]:
 Creating new Local Status db for LocalDav:INBOX.test
Folder INBOX [acc: Davmail]:
 Creating new Local Status db for LocalDav:INBOX
Folder INBOX/test [acc: Davmail]:
 Syncing INBOX/test: IMAP -> MappedIMAP
Folder INBOX [acc: Davmail]:
 Syncing INBOX: IMAP -> MappedIMAP
Folder INBOX/test [acc: Davmail]:
 Copy message 1 (1 of 1) RemoteDav:INBOX/test -> LocalDav
Account sync Davmail:
 *** Finished account 'Davmail' in 0:43

If there is a sub-folder under INBOX, the first run of offlineimap is OK.
The second run will give the error:

shiquanwang@las-z820:~$ offlineimap -a Davmail
OfflineIMAP 6.5.4
  Licensed under the GNU GPL v2+ (v2 or any later version)
Account sync Davmail:
 *** Processing account Davmail
 Establishing connection to localhost:1143
 Establishing connection to localhost:143
 Creating folder INBOX/test[LocalDav]
 ERROR: Creating folder INBOX/test on repository LocalDav
  Folder 'INBOX/test'[LocalDav] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])
 ERROR: Folder 'INBOX/test'[LocalDav] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])
 *** Finished account 'Davmail' in 0:11
ERROR: Exceptions occurred during the run!
ERROR: Creating folder INBOX/test on repository LocalDav
  Folder 'INBOX/test'[LocalDav] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])
ERROR: Folder 'INBOX/test'[LocalDav] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])

streamline repos

This repo seems to be more recent than the one it's forked from, and this issue tracker is pointed to from offlineimap web page. So is this supposed to be the canonical one, or the one at https://github.com/nicolas33/offlineimap ???

Please get rid of the other one if this is the canonical one- otherwise people will think this is just a fork, and the other one is canonical.

Either way, the current situation is confusing to other developers and users. Please advise.

"ctrl +c" hangs on "terminating NOW"

The program doesn't react on signals. E.g. if I press Ctrl+C, it says:

^CTerminating NOW (this may take a few seconds)...

and goes on running. This happens during the FETCH phase.

release on pypi

I could not find offlineimap on pypi, the python package index. Apparently there is no release here. Please make one.

I am willing to help as well & take the newest release & release it on pypi, if that's ok. Just let me know.

Exit Codes on Exception and Errors

When I get a Cert fingerprint error, the exit code of the program is zero, leading to silent failures in automated backups:

OfflineIMAP 6.5.5-rc2
  Licensed under the GNU GPL v2+ (v2 or any later version)
Account sync inf:
 *** Processing account inf
 Establishing connection to REMOVED:993
 ERROR: Server SSL fingerprint '6e848824fc4b80482b432ef779d7c059519b481b' for hostname 'REMOVED' does not match configured fingerprint. Please verify and set 'cert_fingerprint' accordingly if not set yet.
 *** Finished account 'inf' in 0:00
Account sync ile:
 *** Processing account ile
 Establishing connection to REMOVED:993
 ERROR: Server SSL fingerprint 'e499cfa19554e6227b58d10e48b2ee5e73399244' for hostname 'REMOVED' does not match configured fingerprint. Please verify and set 'cert_fingerprint' accordingly if not set yet.
 *** Finished account 'ile' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: Server SSL fingerprint '6e848824fc4b80482b432ef779d7c059519b481b' for hostname 'REMOVED' does not match configured fingerprint. Please verify and set 'cert_fingerprint' accordingly if not set yet.
ERROR: Server SSL fingerprint 'e499cfa19554e6227b58d10e48b2ee5e73399244' for hostname 'REMOVED' does not match configured fingerprint. Please verify and set 'cert_fingerprint' accordingly if not set yet.
xxx@xxx ~ $ echo $?
0

And when the account information is wrong, the exit code is also zero:

[...]
Account sync ile:
 *** Processing account ile
 Establishing connection to REMOVED:993
 ERROR: While attempting to sync account 'ile'
  LOGIN failed.
 *** Finished account 'ile' in 0:02
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'ile'
  LOGIN failed.
xxx@xxx ~ $ echo $?
0

Password safe support.

Hi,

I use a password safe for my gmail account and mutt. E.g.:

set imap_pass=`pwsafe -Ep mutt-$HOST`

Could offlineimap support something like this? Alternatively, if the 'remotepass' option could support arbitrary python code, then a function could easily be written.

After sync messages appears as new

I have the following problem, if I read my emails and save them into sub-folders in my mutt session and then invoke offlineimap sometimes for some emails it will mark them as new in my sub-folders. I'm attaching screen shot of mutt session where I read all emails, but after the sync some of them appeared as new. I use offlineimap version 6.5.5_0 from OSX MacPorts.
Thanks,
Valentin.

--dry-run can drive you nuts

Hi there,

I newly installed offlineimap for testing some archive scenarios and after I finished the config, I just did a --dry-run to test the config and the compatiblity of the used (remote-)IMAP ... and here it started: 4h of debugging a non-existent bug.

Main error from offlineimap was:

offlineimap -o -d IMAP --dry-run -l offlineimap.log

[...]
2013-09-06 00:07:46 WARNING: ERROR: Exceptions occurred during the run!
2013-09-06 00:07:46 WARNING: ERROR: Aborting sync, folder 'INBOX/Alice' [acc: 'licozara']
getfolder() asked for a nonexisting folder 'INBOX/Alice'.
2013-09-06 00:07:46 WARNING:
Traceback:
File "/usr/lib64/python2.7/site-packages/offlineimap/accounts.py", line 372, in syncfolder
replace(remoterepos.getsep(), localrepos.getsep()))
File "/usr/lib64/python2.7/site-packages/offlineimap/repository/Maildir.py", line 134, in getfolder
OfflineImapError.ERROR.FOLDER)

And what was the Problem?
Well, offlineimap never run one "real" shot and so the dir-tree wasnt set up and the dry-run got a lil cold ;)

This case should at least be caught by any exception, to minimze confusion.

Not really a bug, but if I do a dry-run, I kinda expect to get a result as if I do a real run. Also this behavior could lead to hours of confusion ;)

Thanks for reading my 2c.

Lico

Homepage: URL broken

The link to the source git repository in the Documentaiton section on your homepage is broken http://offlineimap.org/#ref-documentation

"source git repository" links to http://github.com/OflineIMAP/offlineimap <- notice the missing f in "OflineIMAP"!

offlineimap in cron with remotepasseval and pythonfile: "Prompting for a password is not supported in this UI backend."

I use a simple Python script to get my IMAP password from Keychain on OS X. If I run offlineimap -u basic from the command line, the configuration below works fine. No user interaction is required. If I put the same command in my crontab, I get the following error in the email report from cron:

OfflineIMAP 6.5.4 Licensed under the GNU GPL v2+ (v2 or any later version) *** Processing account email Establishing connection to imap.server.com ERROR: While attempting to sync account 'email' Prompting for a password is not supported in this UI backend. *** Finished account 'email' in 0:01 ERROR: Exceptions occurred during the run! ERROR: While attempting to sync account 'mpe' Prompting for a password is not supported in this UI backend.

Am I missing something?

Here's the configuration file, with server info changed to protect the innocent.

accounts = email
pythonfile = /usr/bin/get_pw.py

[Account email]
localrepository = local
remoterepository = remote

[Repository local]
type = Maildir
localfolders = ~/.mail/

[Repository remote]
type = IMAP
remotehost = imap.server.com
remoteuser = user
remotepasseval = get_pw("imap.server.com")```

Ask for all account passwords prior to syncing.

If you have several accounts set up, you have to wait for the first account to sync before typing the password for the second account.

It would be better if offlineimap asked for all passwords in one batch.

missing IDLE documentation

I'm trying to get IDLE working on my (gmail) account. The example config file has a brief description of the options, then refers to the manual for details. The manual just points back to the example config file without any further details.

Offlineimap creates additional labels in Gmail and then complains about their UID validity

Problem appeared after upgrading Ubuntu to 12.10 and offlineimap to
6.5.4. It's not particularly severe but gets a little messy.

I have labels in my main Gmail account named after my other e-mail
addresses under another label 'aliasy', for example
aliasy/rafal ochmanski.com. When syncing all labels with offline imap
it correctly creates locally a folder aliasy.rafal ochmanski.com and
downloads the messages but also apparently creates remotely (in the
Gmail account, that is) a folder/label aliasy/rafal ochmanski/com
(note the slash instead of the dot before com) and then complains
about its UID validity https://gist.github.com/f16ec751cccba1eb27cd

I removed those labels (with slashes in place of dots here and there)
but they reappear after offlineimap run. I'm assuming they're being
recreated in the process of syncing.

My config is fairly standard https://gist.github.com/7e5af06945bd8d92470b

indexerror: string index out of range

With latest git clone I receive this error during sync

Thread 'Copy message from Remote:INBOX' terminated with exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/offlineimap/threadutil.py", line 156, in run
Thread.run(self)
File "/usr/lib/python2.7/threading.py", line 504, in run
self.__target(_self.__args, *_self.__kwargs)
File "/usr/local/lib/python2.7/dist-packages/offlineimap/folder/Base.py", line 330, in copymessageto
message = self.getmessage(uid)
File "/usr/local/lib/python2.7/dist-packages/offlineimap/folder/IMAP.py", line 243, in getmessage
data = data[0][1].replace("\r\n", "\n")
IndexError: string index out of range

Last 1 debug messages logged for Copy message from Remote:INBOX prior to exception:

ERROR: 'NoneType' object has no attribute 'group'

Hi,

I'm encountering the following error [1] on one of my email accounts.
I've tried the patch from here [2] but to no avail. Has anyone else
encountered this same problem who might be able to propose a solution or
a work-around.

Thanks,

Footnotes:
[1] ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'unm'
'NoneType' object has no attribute 'group'

[2] http://thread.gmane.org/gmane.mail.imap.offlineimap.general/5642

maildir in a mounted folder raises an errno 20

I'm migrating from a hosting to a VPS, and I need get a backup from mail accounts in hosting to analyze its sizes, etc...

I've mounted a folder in my local PC from a NAS (as storage)...

Then, because use the ":" as part of message filename, maildir.py breaks when do os.open(...

Checked with a maildir in the "local" tree no problem...

If I use maildir-windows-compatible = yes, all is fine...

Tested in Debian 6

IMAPFolder.selectro() should probably have a retry mechanism

Occasionally the Groupwise IMAP server I sync with returns a NO response when offlineimap tries to EXAMINE a folder:

Thread 'Copy message from full:in/review' terminated with exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/offlineimap/threadutil.py", line 156, in run
    Thread.run(self)
  File "/usr/lib64/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/site-packages/offlineimap/folder/Base.py", line 330, in copymessageto
    message = self.getmessage(uid)
  File "/usr/lib/python2.7/site-packages/offlineimap/folder/IMAP.py", line 214, in getmessage
    imapobj.select(self.getfullname(), readonly = True)
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 63, in select
    raise OfflineImapError(errstr, severity)
OfflineImapError: Error SELECTing mailbox 'in/review', server reply:
('NO', ['EXAMINE (C042)'])

It is clear from this error codes reference document that C042 is usually a transitory error caused by another client modifying the same folder. Therefore it would make sense for offlineimap to surround this IMAP operation with a simple exponential backoff / retry mechanism. After a maximum number of retries are exceeded it could then bubble up the exception as per normal.

Attempting NOOP on dropped connection

I am running OfflineIMAP 6.5.5 on OS X. I have launchd configured to start OfflineIMAP in the background. I sync a single IMAP mailbox to a local MailDir.
OfflineIMAP is working fine for some time, but then no more mails arrive. If I take a look at the logs the last entries are always (except the thread numbers of course):

Thread-655:
 Attempting NOOP on dropped connection imap.zoho.com
Thread-657:
 Attempting NOOP on dropped connection imap.zoho.com
Thread-656:
 Attempting NOOP on dropped connection imap.zoho.com

I suspect that it happens when the computer was entering sleep state or the internet connection dropped. Even though OfflineIMAP is still running it never recovers from the connection loss and I have to kill it and start it again.

At first I was using IDLE, but after I read that it is still experimental I thought it might be he culprit here, but the behaviour is still the same after dialling IDLE.

I found some discussion of the NOOP message on some mailing list archive, but that was several years old, so I am not sure if it was ever resolved or if this is a new issue.

If you tell me what you need I am happy to add additional information or debug logs. I would gladly help in resolving this issue, however I have no idea where to start looking on my own…

Crashes, when IMAP folder already exists at destination

I was syncing some accounts with complex IMAP folders. After about 20 folders the script would crash with the error. "Folder already exists."

I only used one connection at a time. So no sync issues, I suggest. Expected behavior would be NOT to creat the folder and just accept that it already exists.

change sslversion to ssl_version in example config

The keyword for the new ssl version feature is 'ssl_version' and not 'sslversion' as shown in the example config file.

doesn't work with Domino IMAP

IBM Lotus Domino version is 8.5.
Dies this way:
offlineimap -l /tmp/p/offlineimap.log -d ALL

[...]
[imap]: 37:10.96 lfs handler _request_pop(BIBN4, ('OK', ['LIST completed'])) = BIBN4
[imap]: 37:10.96 lfs handler LIST:BIBN4.ready.set
Account sync Lotus2Zarafa:
[imap]: 37:10.96 Account sync Lotus2Zarafa _get_untagged_response(LIST) => ['(\Noselect) "\" ""']
[imap]: 37:10.96 Account sync Lotus2Zarafa _untagged_response(OK, ?, LIST) => ['(\Noselect) "\" ""']
ERROR: While attempting to sync account 'Lotus2Zarafa'
'NoneType' object has no attribute 'group'
[' File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 240, in syncrunner\n self.sync()\n', ' File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 302, in sync\n remoterepos.getfolders()\n', ' File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 275, in getfolders\n imapobj = self.imapserver.acquireconnection()\n', ' File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 294, in acquireconnection\n imaputil.imapsplit(listres[0])[1:]\n', ' File "/usr/lib/python2.7/site-packages/offlineimap/imaputil.py", line 148, in imapsplit\n retval.append(m.group('quote'))\n']
*** Finished account 'Lotus2Zarafa' in 0:00
[thread]: Unregister thread 'Account sync Lotus2Zarafa'
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'Lotus2Zarafa'
'NoneType' object has no attribute 'group'

Traceback:
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 240, in syncrunner
self.sync()
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 302, in sync
remoterepos.getfolders()
File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 275, in getfolders
imapobj = self.imapserver.acquireconnection()
File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 294, in acquireconnection
imaputil.imapsplit(listres[0])[1:]
File "/usr/lib/python2.7/site-packages/offlineimap/imaputil.py", line 148, in imapsplit
retval.append(m.group('quote'))

Incremental backup in local folder

There really should be a way to tell offlineimap to backup to the local repository in an incremental way, no local message deletion whatsoever.

There are real use cases in which all one wants to do is to backup the messages from the server and just keep them locally even if they are deleted on the server (low disk space on server, small quotas, etc.)

Maybe a companion "incremental = true" to the "readonly = true" repository option?

too much traffic

Running offline imap with -d ALL resulting in a log file of almost 1G in size. In this file it looks like offlineimap recieves a reply line for every email in any of my inboxes.

 $ offlineimap -o -d ALL -a gmail 2>&1|grep -a "imap.gmail.com reader <"|wc -l
 97512

This seems excessive and would explain why offlineimap spins up my CPU so significantly and takes so long to run. Does the IMAP protocol not provide a way to avoid this exhaustic re-pulling of entire mailboxes on every sync? It looks like there may be a RECENT flag to the EXAMINE command which appears to be causing these huge responses from the server (see http://tools.ietf.org/html/rfc1730#page-16).

IMAP: Flags are not synced?

Just tested it with several accounts: IMAP Flags are not carried over to the other account.

offlineimap confused after (suspend and) resume

I have at (almost) all times offlineimap running in a tmux. After resuming my compu, offlineimap has problems to reconnect properly.
Often it just hangs for a long time (or indefinetely) while nothing seems to happen.

I came up with a solution to the problem by having a resume-script that sends SIGUSR2 to my running offlineimap, whereafter it is automatically restarted.

The problem though, is that sometimes offlineimap takes a long time to exit upon SIGUSR2. Maybe this is again related to the lost tcp connections, timeouts, and such. And essentially the same problem as above.

Any suggestions on how I should solve? I have been reluctant to just send SIGKILL to the process, because I was afraid that this might cause inconsistencies in the repo, or such. But maybe this is what I need and should to do?

exceptions not handled correctly in single-threaded mode

As described in this post.

@konvpalto mentioned in #29 that he can't reproduce, so I guess I need to provide a test case for this. I don't have time right now, but I'll file this bug here so that we don't forget about it.

Invoke folderfilter at each synchronization

At the moment, folderfilter is only invoked once per folder, when offlineimap is launched. However, invoking folderfilter each time a synchronization is due, this would give the user the ability to have a "dynamic" filter function. In my case, I would use this feature to synchronize the Junk folder only once per hour by returning "False" in the filter function in case that Junk was already synchronized in the current hour.

corrupt mail resulting from use of IDLE (IMAP->Maildir)

I've been happily using offlineimap 6.5.5 to do IMAP->Maildir for weeks. Just today, I configured it to do IDLE by adding:

idlefolders = ['INBOX']

The next run of offlineimap told me:

Account sync SangerZuul:
*** Finished account 'SangerZuul' in 0:05
Next refresh in 5.0 minutes
Thread-9:
Syncing INBOX: IMAP -> Maildir
Copy message 771539 (1 of 1) SangerZuul-remote:INBOX -> SangerZuul-local

And now mutt is showing me a message at the top of my inbox, apparently dated Jan 01, no subject, 0K size. This is what the message actually looks like in the Maildir:

9dh3@no:~/offlineimap/sanger.ac.uk/INBOX/new$ ls -l 
-rw-rw-r-- 1 dh3 dh3 1 Nov 13 14:25 1384352744_0.25906.no,U=771540,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
dh3@no:~/offlineimap/sanger.ac.uk/INBOX/new$ cat 1384352744_0.25906.no,U=771540,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
9dh3@no:~/offlineimap/sanger.ac.uk/INBOX/new$ 

(i.e. the file contains a single digit "9", not a properly-formatted message)

It looks like quite a coincidence that the thread number in the sync output is the same as the errant digit that's ended up in the message file.

I'll take a look but I've never examined the offlineimap source before, and I'm no IMAP expert...

Cheers,
Dave