with -a CLI option, mbnames must not erase the entries of other accounts

[offlineimap]

with -a CLI option, mbnames must not erase the entries of other accounts

[nicolas33]

Yes! Exactly this.

Ok, thanks for your insistence. I did not understand your point soon because I was focused on how the file is constructed instead of the result. It's obvious that -a must not erase the entries of the other accounts. The solving is easy but we definitely won't connect to the other remotes.

So, mbnames should build intermediate mbnames in <metadata>/mbnames in a per-account basis. At the end of the sync of each account, the intermediate file is updated.

Then, the real target can be built by concatenation of all the intermediate mbnames. This operation is triggered at the end of the full sync if incremental option is disabled and at the end of the sync of each account otherwise.

We might want to add a purge option in the mbnames section to remove any intermediate file not in the accounts list.

[nicolas33]

I'm not using mbnames myself but I agree that having this outside any account section or outside the general section is wrong.

Is it possible to workaround this issue by using the sort_keyfunc?

[nicolas33]

I don't get why mbnames would have to be constructed for accounts not being synced.

[lahwaacz]

Because mailbox names are global file (as expected by e.g. mutt), so running offlineimap should always produce the same file, regardless of which accounts are being synced. I strongly recommend to reopen this bug.

[nicolas33]

Please, provide a sample of the current behaviour and what you would expect.

[lahwaacz]

offlineimap's config snippet:

[mbnames]
enabled = yes
filename = ~/mailboxes
header = "mailboxes "
peritem = "+%(accountname)s/%(foldername)s"
sep = " "
footer = "\n"

[Account foo]
...

[Account bar]
...

Testing current behaviour:

$ offlineimap -o -u quiet -a foo
$ cat ~/mailboxes 
mailboxes "+foo/INBOX" "+foo/drafts" "+foo/sent" "+foo/spam" "+foo/trash"
$ offlineimap -o -u quiet -a bar
$ cat ~/mailboxes 
mailboxes "+bar/INBOX" "+bar/drafts" "+bar/sent" "+bar/spam" "+bar/trash"

Expected result for both cases:

mailboxes "+bar/INBOX" "+bar/drafts" "+bar/sent" "+bar/spam" "+bar/trash" "+foo/INBOX" "+foo/drafts" "+foo/sent" "+foo/spam" "+foo/trash"

[nicolas33]

Why not run offlineimap -o -u quiet?

[nicolas33]

Actually, this would be wrong to make any wite for accounts not requested at CLI.

[lahwaacz]

Why not run offlineimap -o -u quiet?

For the same reason why quick specificaton of accounts on the command-line exists in the first place - to focus on doing the most important task as fast as possible. Syncing all accounts at once might take a long time, while users might need to quickly sync one specific account, for example via hooks for currently open mailbox in mutt.

Actually, this would be wrong to make any wite for accounts not requested at CLI.

I could understand a problem with not respecting the accounts list in the [general] section of the config, but I don't see a problem with taking all accounts from the config regardless of the CLI options. Please elaborate why this is such a big problem.

[nicolas33]

As soon as we would implement writes for accounts not requested at CLI, users will complain about unexpected directory creations.

I still don't get exactly why you need the creation of (empty) folders for maildirs not going to be synced.

I wonder this use case is too much specific and does not worth the code complexity to be added for this edge-case feature.

Please, don't get me wrong. I'm not offending you and I'm not invalidating your use case. I'm pointing out that this specific use case is marginal enough and would introduce unexpected behaviour for most users to worth the change.

[lahwaacz]

The expected behaviour is not that all the directories would be created, but they would be written to the mailbox names file (set with the filename option in [mbnames]). It is the client's responsibility to handle non-existent directories in a sane way; in my use case all directories already exist due to previous sync of all the accounts.

[nicolas33]

Yes, sorry for not being clear. Writing mbnames to the filename for non-synced accounts is wrong because entries would be created for non-existing folders. mbnames is supposed to map to the local maildir of the synced folders.

We are NOT supposed to connect to a remote account if we are not requested to sync it.

in my use case all directories already exist due to previous sync of all the accounts.

So, should be the mbname entries.

[lahwaacz]

mbnames is supposed to map to the local maildir of the synced folders.

I think this is where we'll disagree. From the clients' point of view, e.g. Mutt reading the mbnames file, it expects all folders that currently exist locally, even if they are not synced most recently. Now it's up to offlineimap to ensure that this expectation is fulfilled in all its operation.

We are NOT supposed to connect to a remote account if we are not requested to sync it.

Now I can see why the implementation seems too complex to you. I agree that connecting to remote accounts that are not being synced is wrong, but you don't have to do that for this functionality. For example, the last-synced folders are already cached in the Repository-<name>-local/FolderValidity/<folder name> files. For accounts that are not being synced, this is a read-only operation.

[dolohow]

What about switch --create-mbnames?

And also I agree with @nicolas33, we should not create directories of other accounts when syncing only specific with -a switch.

[nicolas33]

From the clients' point of view, e.g. Mutt reading the mbnames file, it expects all folders that currently exist locally, even if they are not synced most recently.

At the end of the sync, the mapping is updated. We can't write the mbnames before we know the local maildirs to create. We know this at sync time.

the last-synced folders are already cached in the Repository--local/FolderValidity/ files. For accounts that are not being synced, this is a read-only operation.

I don't get why it is required to update the mbnames after the lastest sync since it was already updated at the end of the latest sync.

If this feature only requires to read the local cache, I think a little shell/lua/whatever script can easily do the job. Hence, I'm still not convince we should implement this in offlineimap. I'd be delighted that any user request us to add its own script in the contrib/ directory of the official sources. Even if I don't get how it's usefull.

[nicolas33]

Hmm, I guess I'm getting your issue. When used with -a, is offlineimap erasing or clearing the entries of the other accounts?

[lahwaacz]

Hmm, I guess I'm getting your issue. Is offlineimap erasing or clearing the entries of the other accounts when used with -a?

Yes! Exactly this. Please take a look at this post again, it's all there: after offlineimap -o -u quiet -a foo the mbnames file contains entries only for the foo account, regardless of the previous content, sync or anything. That means that you can't effectively use the -a option with different accounts along with the [mbnames] section enabled, because the writes to the mbnames file will overwrite each other.

When you've finally got it, can this issue be finally reopened to acknowledge it as a valid bug?

[nicolas33]

@lahwaacz Could you test https://github.com/nicolas33/offlineimap/tree/ns/mbnames please?

[nicolas33]

I did some basic tests, here.

Notice you don't need to create folders to check updates are done right. Instead, you might want to check the date and content of the files in the cache (<metadata>/mbnames) and the content of your mbnames target file.

[lahwaacz]

I think that will fix the issue. Is there a specific reason to use yaml and require a 3rd party module instead of using e.g. json from the standard library?

[dolohow]

Is there a specific reason to use yaml and require a 3rd party module instead of using e.g. json from the standard library?

That is a good question.

[nicolas33]

Is there a specific reason to use yaml and require a 3rd party module instead of using e.g. json from the standard library?

That is a good question.

I second that. I didn't know json is in the standard library. Will change that soon. Good catch!

[nicolas33]

https://github.com/nicolas33/offlineimap/tree/ns/mbnames-2

[nicolas33]

Fixed.