postalsys/imapflow

IMAP Client library for EmailEngine Email API (https://emailengine.app)

ImapFlow

ImapFlow is a modern and easy-to-use IMAP client library for Node.js.

Note

Managing an IMAP connection is cool, but if you are only looking for an easy way to integrate email accounts, then ImapFlow was built for EmailEngine Email API. It's a self-hosted software that converts all IMAP accounts to easy-to-use REST interfaces.

The focus for ImapFlow is to provide easy to use API over IMAP. Using ImapFlow does not expect knowledge about specific IMAP details. A general understanding is good enough.

IMAP extensions are handled in the background, so, for example, you can always request labels value from a {@link FetchQueryObject|fetch()} call, but if the IMAP server does not support X-GM-EXT-1 extension, then labels value is not included in the response.

Source

Source code is available from Github.

Usage

First install the module from npm:

npm install imapflow

next import the ImapFlow class into your script:

const { ImapFlow } = require('imapflow');

Promises

All ImapFlow methods use Promises, so you need to wait using await or wait for the then() method to fire until you get the response.

const { ImapFlow } = require('imapflow');
const client = new ImapFlow({
    host: 'ethereal.email',
    port: 993,
    secure: true,
    auth: {
        user: '[email protected]',
        pass: 'mW6e4wWWnEd3H4hT5B'
    }
});

const main = async () => {
    // Wait until client connects and authorizes
    await client.connect();

    // Select and lock a mailbox. Throws if mailbox does not exist
    let lock = await client.getMailboxLock('INBOX');
    try {
        // fetch latest message source
        // client.mailbox includes information about currently selected mailbox
        // "exists" value is also the largest sequence number available in the mailbox
        let message = await client.fetchOne(client.mailbox.exists, { source: true });
        console.log(message.source.toString());

        // list subjects for all messages
        // uid value is always included in FETCH response, envelope strings are in unicode.
        for await (let message of client.fetch('1:*', { envelope: true })) {
            console.log(`${message.uid}: ${message.envelope.subject}`);
        }
    } finally {
        // Make sure lock is released, otherwise next `getMailboxLock()` never returns
        lock.release();
    }

    // log out and close connection
    await client.logout();
};

main().catch(err => console.error(err));

Documentation

API reference.

License

© 2020-2024 Postal Systems OÜ

Licensed under MIT-license

imapflow's Issues

Logger documentation is incorrect

Describe the bug
The documentation suggests that the logger handler must support debug,warn, info, and error levels. This is incorrect.

To Reproduce
Create a custom logger:

class CustomLogger {
    constructor() {
        autobind(this);
    }

    debug(obj) {
    }

    info(obj) {
    }

    warn(obj) {
    }

    error(obj) {
    }
}```

**Expected behavior**
Given the above logger, all logging input should be squelched.

**Screenshots**
An error occurs: `mainLogger[level] is not a function` as it steps through trace and fatal as levels as well.

**Desktop (please complete the following information):**
 - OSX
 - NodeJS 12

**Additional context**
Easiest fix is to change the documentation to specify the requirement for trace and fatal; another fix would be to fail *gently* (i.e., have a fallback) but the documentation is the easiest thing to fix.

Gmail disconnects randomly, client can not be reused

This is a suggestion as much as it is a bug, but here it goes:

When Gmail closes a connection randomly even though I am IDLEing, that is fine. I get a close event.

The problem is when I try to then client.connect() immediately or after 10 seconds, the client object must have some old connection data in it and I receive:

08:39:54/Error: {"err":{"code":"ERR_STREAM_WRITE_AFTER_END"},"cid":"xxxxx"}
events.js:298
throw er; // Unhandled 'error' event
^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
at writeAfterEnd (_stream_writable.js:271:14)
at ImapStream.Writable.write (_stream_writable.js:320:5)
at TLSSocket.ondata (_stream_readable.js:708:22)
at TLSSocket.emit (events.js:321:20)
at addChunk (_stream_readable.js:297:12)
at readableAddChunk (_stream_readable.js:273:9)
at TLSSocket.Readable.push (_stream_readable.js:214:10)
at TLSWrap.onStreamRead (internal/stream_base_commons.js:186:23)
Emitted 'error' event on ImapFlow instance at:
at ImapStream. (/home/jschoof/FTP/CCServer/CarCords/node_modules/imapflow/lib/imap-flow.js:247:18)
at ImapStream.emit (events.js:333:22)
at errorOrDestroy (internal/streams/destroy.js:128:12)
at writeAfterEnd (_stream_writable.js:273:3)
at ImapStream.Writable.write (_stream_writable.js:320:5)
[... lines matching original stack trace ...]
at TLSSocket.Readable.push (_stream_readable.js:214:10) {
code: 'ERR_STREAM_WRITE_AFTER_END'
}

I have compression disabled...

So my solution has been to rebuild the client object from scratch with another client = new ImapFlow so that client is fresh. Is this the way you wanted it to work andris9? Maybe your client.connect() code could make sure the client object is "clean" so it is fresh and ready for another client.connect() after an unknown close? Just a suggestion.

Great Library! Thanks so much! I'm using this to receive text commands, that are turned into email, through google IMAP, parsed and then sent to our new Electric Vehicle Charging Stations so they can be controlled and managed through texts as well as web apps. Some people can text but have no or limited data connectivity. Your ImapFlow will be mission critical!

Timeout for `client.getMailboxLock()` when there is a network disconnection

If connection goes down while doing client.getMailboxLock('INBOX'),
I noticed that the execution of that function gets blocked for 15 or
16 minutes; after that, the ENETUNREACH error is raised.

This behaviour can be tested with a minimal working example like the
code I posted in #47,
running the code and then disconnecting the NetworkManager client.
I tested this with Debian testing and node v.12.

Is there a way to shorten that timeout?

ETIMEDOUT on IDLE

When the client is IDLEing, I get ETIMEDOUT errors:

{
  err: Error: read ETIMEDOUT
      at TLSWrap.onStreamRead (internal/stream_base_commons.js:201:27) {
    errno: 'ETIMEDOUT',
    code: 'ETIMEDOUT',
    syscall: 'read'
  }
}

The log shows a bit more detail (plus timings):

  imap:server debug 4C IDLE +15s
  imap:server debug + idling +87ms
  imap:client debug initiated IDLE +0ms
  imap:client debug breaking IDLE +5m
  imap:other error {
  err: Error: read ETIMEDOUT
      at TLSWrap.onStreamRead (internal/stream_base_commons.js:201:27) {
    errno: 'ETIMEDOUT',
    code: 'ETIMEDOUT',
    syscall: 'read'
  }
} +58s
  imap:other error {
  err: Error: read ETIMEDOUT
      at TLSWrap.onStreamRead (internal/stream_base_commons.js:201:27) {
    errno: 'ETIMEDOUT',
    code: 'ETIMEDOUT',
    syscall: 'read'
  }
} +3ms

(Using a custom logger with debug for the timings; imap:server = S:, imap:client = C:)

This appears to be linked to the socket timeout in some non-obvious way; decreasing to 2 minutes instead of 5 makes the IDLE -> NOOP -> IDLE loop work properly. It seems like when it writes the DONE, it's timing out waiting for the tagged response. This could be a Gmail-specific thing perhaps.

Still digging into what's happening here.

Requesting reconnect keeps old connection open

If connection reconnect is requested then old connection is not closed.

[Question] Is it possible to fetch messages in a descending order?

I would love to implement a way to show emails as in gmail, last received is first shown. So I needed the feature to fetch emails in using pagination.
the way I tried to implement it is by implementing my own descending fetch, and running a regex on the subject:

/**
   * Generator for fetching an email starting from the most recent one
   * This generator generates the latest email that respects a subject regex
   * @param {number} start - uid from where to start getting the emails in a descending order
   * @param {string} subject - regex subject of the email that needs to be respected 
   * @return {AsyncGenerator<null, void, *>}
   */
async* fetchDescending (start, subject) {
    let nextEmailUid = start

    // returning the email using the nextEmailUid
    while (nextEmailUid > 0) {
      // is it possible to pass some searching criteria to fetchOne ? 
      let nextEmail = await client.fetchOne(nextEmailUid, { envelope: true, uid: true }, { uid: true } )

      // Yielding the email or marking it as read
      if (nextEmail) {
        // returning it only if it has the right Subject type!
        if (extractSubject(nextEmail.envelope.subject) === subject) {
          yield nextEmail
        }
      }

      // decrementing the email uid and resetting the nextEmail to null
      nextEmailUid --
    }
  }

doing it this way is very slow, for an inbox of 80 mails it takes about 6 seconds...
my question is the following:

• Can I fetch emails in a descending manner something like fetch("*-10:*") to fetch the last 10 emails?
• Is it possible to fetch emails that respect a certain regex?

thank you very much @andris9 and other contributors for this very useful module.

Unable to add \\Seen flag

I'm facing an issue similar to #3 while trying to mark all messages from a given thread as Seen:

await client.messageFlagsAdd({ threadId: req.query.threadId }, ["\\Seen"])

{"level":30,"time":1594658321913,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"connection","msg":"Established secure TCP connection","cid":"sklianrmai9929bqotja","secure":true,"host":"imap.gmail.com","servername":"imap.gmail.com","port":993,"address":"108.177.15.108","localAddress":"192.168.104.31","localPort":47580,"authorized":true,"algo":"TLS_AES_256_GCM_SHA384","version":"TLSv1.3"}
{"level":20,"time":1594658321963,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* OK Gimap ready for requests from 109.119.144.165 z15mb7838744wmk","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658321963,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"1 CAPABILITY","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322013,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN AUTH=OAUTHBEARER AUTH=XOAUTH","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322013,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"1 OK Thats all she wrote! z15mb7838744wmk","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322014,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"2 ID (\"name\" \"imapflow\" \"version\" \"1.0.47\" \"vendor\" \"IMAP API\" \"support-url\" \"https://github.com/andris9/imapflow/issues\")","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322064,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* ID (\"name\" \"GImap\" \"vendor\" \"Google, Inc.\" \"support-url\" \"http://support.google.com/mail\" \"remote-host\" \"109.119.144.165\" \"connection-token\" \"z15mb7838744wmk\")","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322064,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"2 OK Success z15mb7838744wmk","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322064,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"3 AUTHENTICATE OAUTHBEARER \"(* value hidden *)\"","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322860,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658322860,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"3 OK [email protected] authenticated (Success)","cid":"sklianrmai9929bqotja"}
{"level":30,"time":1594658322860,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"auth","msg":"User authenticated","cid":"sklianrmai9929bqotja","user":"[email protected]"}{"level":20,"time":1594658322860,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"4 NAMESPACE","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323060,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* NAMESPACE ((\"\" \"/\")) NIL NIL","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323060,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"4 OK Success","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323060,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"5 COMPRESS DEFLATE","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323212,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"5 OK Success","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323212,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"6 ENABLE CONDSTORE UTF8=ACCEPT","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323374,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* ENABLED CONDSTORE UTF8=ACCEPT","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323374,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"6 OK Success","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323374,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"7 LIST \"\" \"INBOX\"","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323543,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* LIST (\\HasNoChildren) \"/\" \"INBOX\"","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323543,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"7 OK Success","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323543,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"8 LSUB \"\" \"INBOX\"","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323694,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* LSUB (\\HasNoChildren) \"/\" \"INBOX\"","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323694,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"8 OK Success","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323694,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"9 SELECT INBOX","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323856,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* FLAGS (\\Answered \\Flagged \\Draft \\Deleted \\Seen $Forwarded $Junk $MDNSent $MailFlagBit0 $MailFlagBit1 $MailFlagBit2 $NotJunk $NotPhishing $Phishing JunkRecorded NonJunk NotJunk)","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323856,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* OK \"\"[PERMANENTFLAGS (\\Answered \\Flagged \\Draft \\Deleted \\Seen $Forwarded $Junk $MDNSent $MailFlagBit0 $MailFlagBit1 $MailFlagBit2 $NotJunk $NotPhishing $Phishing JunkRecorded NonJunk NotJunk \"\\\\*\")] Flags permitted.","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323856,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* OK \"\"[UIDVALIDITY 3] UIDs valid.","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323856,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* 241 EXISTS","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323856,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* 0 RECENT","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323857,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* OK \"\"[UIDNEXT 75216] Predicted next UID.","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323857,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* OK \"\"[HIGHESTMODSEQ 10165785]","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323857,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"9 OK \"\"[READ-WRITE] INBOX selected. (Success)","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658323857,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"A UID SEARCH X-GM-THRID 1672013625230246191","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658324020,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* SEARCH 75197 75210","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658324020,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"A OK SEARCH completed (Success)","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658324020,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"B STORE 75197,75210 +FLAGS (\\Seen)","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658324180,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"B OK Success","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658324180,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"C LOGOUT","cid":"sklianrmai9929bqotja"}
{"level":20,"time":1594658324330,"pid":30084,"hostname":"OmegaAssembled","component":"imap-connection","cid":"sklianrmai9929bqotja","src":"s","msg":"* BYE LOGOUT Requested","cid":"sklianrmai9929bqotja"}

Everything looks fine here, then when I fetch the messages again, the \Seen flag is not there (yes, I'm settings flags: true in the fetch options).

Thank you!

Unable to catch the ECONNRESET error

I am not able to catch the ECONNRESET error.

I have prepared a stripped down version of my code (since the error is
not easily replicable, also the code I am actually using is large)
and it is shown below.

In there, I simulate errors emitted by client and they are caught (see commented lines).
But when an actual ECONNRESET occurs, it is not caught (and the server crashes).

'use strict';

const pino = require('pino')();
pino.level = 'silent';
//pino.level = 'info';

const fs = require('fs');
const { ImapFlow } = require('imapflow');

//const rawConfigData = JSON.parse(fs.readFileSync('imapGmailConfig.json'));
const rawConfigData = JSON.parse(fs.readFileSync('imapPecConfig.json'));
rawConfigData.logger = pino;

const main = async () => {
  for (let i = 0; i < 50; i++) {
    let lock;
    let client;
    try {
      console.log("new ImapFlow");
      client = new ImapFlow(rawConfigData);
      await client.connect();

      lock = await client.getMailboxLock('INBOX');
      
      console.log("INFO: start fetching message");
      let msg = await client.fetchOne('*', { envelope: true });

      // error simulation here --> gets caught by the catch below with console.error("ERROR here1:", err);
      //
      // if (i==2) {
      //   client.emit('error', new Error('ECONNRESET'));
      // }
      
    } catch(err) {
      console.error("ERROR here1:", err);
      
    } finally {
      console.log("releasing: lock.release()");
      lock.release();
    }

    // another error simulation here --> gets caught by the catch
    // below at the line with main().catch(err => console.error("ERROR here0: ", err));
    //
    // if (i==4) {
    //   client.emit('error', new Error('ECONNRESET'));
    // }
      
    console.log("await client.logout():");
    await client.logout();
  }
};

main().catch(err => console.error("ERROR here0: ", err));

Actual ECONNRESET error is like this:

events.js:291
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:209:20)
Emitted 'error' event on ImapFlow instance at:
    at ImapFlow.emitError (/home/giacomo/imapflow_examples/node_modules/imapflow/lib/imap-flow.js:257:14)
    at TLSSocket._socketError (/home/giacomo/imapflow_examples/node_modules/imapflow/lib/imap-flow.js:495:22)
    at TLSSocket.emit (events.js:326:22)
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  errno: 'ECONNRESET',
  code: 'ECONNRESET',
  syscall: 'read'
}

I am using imapflow version 1.0.57.
How can I improve my code to catch ECONNRESET?

Any help is very appreciated, thanks.

---

EDIT: It seems that the error is raising (mostly) when trying to do client.logout().

Download

Cant seem to download mails using either uid or sequence string

Memory leak?

Describe the bug

I'm using ImapFlow to login into a mail box, and in a long-running process fetch each new mail.
After some time on running, it seems that node process is taking more and more memory.
After a bit of debugging, it seems that at some point ImapFlow.requestTagMap is becoming very big (>50000 items inside)

Here is the way I fetch messages:

for await (const msg of imapflow.fetch(pattern, { envelope: true, bodyStructure: true, bodyParts: ['text'] })) {
 // process message
}

Environment

• OS: linux
• Node: v14.17.6

fetch limit not working in traditional web-mail server

Describe the bug

While I try to fetch unseen mail from inbox, fetch limit working properly for Gmail, but the same code snippet not working while I try to fetch mail from the traditional hosting providers Web-Mail server. Fetch results always empty.

Code sample

 for await (let message of client.fetch('1:20', { envelope: true, source: true, unseen: true })) {
     console.log(`${message.uid}: ${message.envelope.subject}`);
 }

Note: When I use fetch('1:*', {}) then it woking but fetch('1:20', {}) not working properly.

mainLogger[level] is not a function

Describe the bug
When I use a custom logger with the methods debug(obj), info(obj), warn(obj) and error(obj) than this error is thrown:

TypeError: mainLogger[level] is not a function
    at Object.synteticLogger.<computed> [as info] (node_modules/imapflow/lib/imap-flow.js:2262:34)
    at TLSSocket.<anonymous> (node_modules/imapflow/lib/imap-flow.js:1044:26)
    at Object.onceWrapper (events.js:416:28)
    at TLSSocket.emit (events.js:310:20)
    at TLSSocket.EventEmitter.emit (domain.js:482:12)
    at TLSSocket.onConnectSecure (_tls_wrap.js:1498:12)
    at TLSSocket.emit (events.js:310:20)
    at TLSSocket.EventEmitter.emit (domain.js:482:12)
    at TLSSocket._finishInit (_tls_wrap.js:917:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:687:12)

To Reproduce
Steps to reproduce the behavior:

1. Install version 1.0.47
2. Create an instance with a custom logger with the methods debug(obj), info(obj), warn(obj) and error(obj)

Desktop (please complete the following information):

• OS: linux docker image node:12-alpine
• Browser none
• Version 1.0.47

Commands do never return or throw after connection troubles

Description

Connection troubles can make command-execution to never return code-flow to the calling code (including throwing exceptions).

To Reproduce

Code:

await client.getMailboxLock('INBOX');

Turn off the internet while a command is running. The client will emit an event-error, but the runnining command will never finished.

Expected behavior

Running command should throw the same error as EventEmitter to give control back. Command-timeouts should be set too.

Environment

• OS: Linux, Ubuntu 20.04
• NodeJS: v14.9.0
• imapflow: v1.0.48

Probable reason

ImapFlow waits for response with the same token as the request. But the response is lost because of connection troubles.

Probable solution

Maybe we can listen for errors while command is running and reject the promise if an error occurred?
Also we should set command-timeouts.

Temporary solution in user-code

async function _wrapOperation(thisArg, operationFunc, timeout, parameters = []) {
  let stoppableEventsResolveHandler = null;
  let stoppableEventsErrorHandler = null;
  let stoppableEventsCloseHandler = null;

  const stoppableEventsPromise = new Promise((resolve, reject) => {
    stoppableEventsResolveHandler = resolve;
    stoppableEventsErrorHandler = reject;

    stoppableEventsCloseHandler = () => {
      reject(new Error(`Unexpected client close happened while executing ${operationFunc.name}`));
    };

    _client.once('error', reject);
    _client.once('close', stoppableEventsCloseHandler);
  });

  const operationPromise = Promise.resolve().then(() => {
    return operationFunc.call(thisArg, ...parameters);
  });

  let timeoutHandler = null;
  const timeoutPromise = new Promise((_, reject) => {
    const timeoutMessage = `Timeout ${timeout}ms reached while executing ${operationFunc.name}`;
    timeoutHandler = setTimeout(() => reject(new Error(timeoutMessage)), timeout);
  });

  await Promise.race([operationPromise, stoppableEventsPromise, timeoutPromise]).finally(() => {
    clearTimeout(timeoutHandler);
    _client.removeListener('error', stoppableEventsErrorHandler);
    _client.removeListener('close', stoppableEventsCloseHandler);
    stoppableEventsResolveHandler();
  });

  return operationPromise;
}

// ...

const lock = await _wrapOperation(_client, _client.getMailboxLock, OPERATION_TIMEOUT, ['INBOX']);
// ...

The temporary solution breaks logout-command as described in #28. Use client.close() directly, clear all resources and create a new client.

Client hangs on logout when there is no connection to server

Version
imapflow 1.0.56

Behavior:
If the client fails to connect to server for some reason, then await client.logout() will hang indefinitely

Expected behavior:
The method should detect there was no connection to logout from in the first place, or it should reject the promise after a reasonable amount of time (eg. using Promise.race)

Sample code:

const {ImapFlow} = require("imapflow");

const runMailTask = async () => {
    const client = new ImapFlow({
        host: "not.a.real.dns.name.example"
    });

    try {
        await client.connect();
    } catch (err) {
        console.error("ERROR:", err)
    } finally {
        try {
            await client.logout();
            console.log("Logged out") // <- code never reaches here
        } catch (err) {
            console.error("ERROR: (logout):", err) // <- code never reaches here
            client.close();
        }
    }
};

(async function () {
    await runMailTask();
    console.log("Finished mail task") // <- code never reaches here
}())

Output:

{"level":50,"time":1619772740057,"pid":10854,"hostname":"shunkica","component":"imap-connection","cid":"46jsai1b3yunt8rr5w1j","err":{"type":"Error","message":"getaddrinfo ENOTFOUND not.a.real.dns.name.example","stack":"Error: getaddrinfo ENOTFOUND not.a.real.dns.name.example\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26)","errno":-3008,"code":"ENOTFOUND","syscall":"getaddrinfo","hostname":"not.a.real.dns.name.example"},"cid":"46jsai1b3yunt8rr5w1j"}
ERROR: Error: getaddrinfo ENOTFOUND not.a.real.dns.name.example
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26) {
  errno: -3008,
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'not.a.real.dns.name.example'
}
{"level":20,"time":1619772740065,"pid":10854,"hostname":"shunkica","component":"imap-connection","cid":"46jsai1b3yunt8rr5w1j","src":"s","msg":"1 LOGOUT","cid":"46jsai1b3yunt8rr5w1j"}

To handle this I need to do something like this which seems convoluted for such a simple task:
await Promise.race([client.logout(), new Promise((_, reject) => setTimeout(() => reject("Timed out"), 5000))]);

Export types in package

Hi! I'm trying to build some functions which use return values from parts of imapflow, but the types aren't exported from the package.

It's possible to work around this with a bit of magic using the function return types:

import { ImapFlow } from 'imapflow';

type PromiseResult<T> = T extends PromiseLike<infer U> ? U : T;
export type FetchMessageObject = PromiseResult<ReturnType<ImapFlow['fetchOne']>>;

Would it be possible to have the types exported directly?

Proxy support

Would like to see the proxy support functionality, as none of the other nodejs imap clients have it

Example:

const client = new ImapFlow({
    host: 'imap.somehost.com,
    port: 993,
    secure: true,
    auth: {
        user: '[email protected],
        pass: '1q2wedrf'
    },
    proxy: {
        host: '1.1.1.1', 
        port: 12345,
        type: 'http',              // http/s  socks4  socks5
        username: 'user', 
        password: 'pass' 
    }
});

Describe alternatives you've considered
imap, imap-simple, imap-simple-with-proxy (no http/s proxy support), emaljs-imap-client

Imapflow stops listening for new imap events after while

Hi, and thank you for the amazing library!

I have an issue which is difficult to reproduce.

Imapflow version: 1.0.39, same behaviour with older versions as well

In my app I'm listening for exists event to check for a new mail.
It works fine, but after few days (sometime few hours) exists event stops firing with no sign that something went wrong.
I suspect that connection breaks, without being detected. (no error event firing)
If I restart my app it works again.

Log output:

{"level":20,"time":1586405156088,"pid":18,"hostname":"49d844c5f708","component":"imap-connection","cid":"ph1z3tufntqm37xppwxb","src":"s","msg":"* OK Still here","cid":"ph1z3tufntqm37xppwxb"}
{"level":20,"time":1586405276167,"pid":18,"hostname":"49d844c5f708","component":"imap-connection","cid":"ph1z3tufntqm37xppwxb","src":"s","msg":"* OK Still here","cid":"ph1z3tufntqm37xppwxb"}
{"level":20,"time":1586405396284,"pid":18,"hostname":"49d844c5f708","component":"imap-connection","cid":"ph1z3tufntqm37xppwxb","src":"s","msg":"* OK Still here","cid":"ph1z3tufntqm37xppwxb"}
{"level":20,"time":1586405459550,"pid":18,"hostname":"49d844c5f708","component":"imap-connection","cid":"ph1z3tufntqm37xppwxb","src":"c","msg":"breaking IDLE"}

Is there something I can do to check if connection still alive?

Question: syntax of 'Date' operand in messageDelete({before: Date})

Thank you very much for this really fine piece of work; most helpful.

I cannot get messageDelete to work with the before SearchObject property.

I set the value of before to a new Date() which represents, say, an hour ago. There are definitely messages in the inbox that were received prior to that.

Where your documentation says 'Date' does that mean a JavaScript Date?

Also, what is the correct format for a date operand expressed as a string?

Thanks again!

-Paul

"exists" event handler not being invoked

Describe the bug
It looks like there is an issue with "exists" handler invocation, whenever a new email arrives to a subscribed mailbox

To Reproduce
I attached my backend log with detailed descriptions of significant steps https://gist.github.com/phi1ipp/605b519f9d7f2a035461596d82fcc848

Expected behavior
Whenever a new email arrives in the mailbox, which a client is subscribed on, there should be an invocation of "exists" handler

I'd appreciate if you can provide a clue for a possible issue in my processing, but based on the detailed logger info from the library, server responds as expected, yet client library does not process the events accurately.

Thank you in advance!

unexpected end of file error when logout called

Node v10.16.3
OS Linux

When calling
await client.logout();
I am using the example code.
This error is thrown, included other messages prior, to show logout is successful.

{"level":20,"time":1594996301712,"pid":3454774,"hostname":"xxxx","component":"imap-connection","cid":"1a9wjst60odltvjz1a4p","src":"s","msg":"A OK LOGOUT completed","cid":"1a9wjst60odltvjz1a4p"}

{"level":50,"time":1594996301713,"pid":3454774,"hostname":"xxxx","component":"imap-connection","cid":"1a9wjst60odltvjz1a4p","err":{"type":"Error","message":"unexpected end of file","stack":"Error: unexpected end of file\n at Zlib.zlibOnError [as onerror] (zlib.js:162:17)","errno":-5,"code":"Z_BUF_ERROR"},"cid":"1a9wjst60odltvjz1a4p"}
events.js:174
throw er; // Unhandled 'error' event
^

Error: unexpected end of file
at Zlib.zlibOnError [as onerror] (zlib.js:162:17)
Emitted 'error' event at:
at ImapStream.ImapFlow.streamer.on.err (/.../node_modules/imapflow/lib/imap-flow.js:247:18)
at ImapStream.emit (events.js:203:15)
at InflateRaw._inflate.on.err (/.../node_modules/imapflow/lib/imap-flow.js:584:27)
at InflateRaw.emit (events.js:198:13)
at Zlib.zlibOnError [as onerror] (zlib.js:165:8)

IMAP-syntax error, UID-sequence mustn't be quoted in search-command

Description

searchCompiler generates syntax-incorrect command, when a sequence used in uid-argument.

To Reproduce

Code:

await client.search({uid: '3:*'});

searchCompiler outputs the following IMAP-command:

A SEARCH UID "3:*"

This is not correct, server responds: A BAD Could not parse command

Expected behavior

searchCompiler must output the following IMAP-command:

A SEARCH UID 3:*

Environment

• OS: Linux, Ubuntu 20.04
• NodeJS: v14.9.0
• imapflow: v1.0.48

The reason

This happens because of uid-argument's type in search-command ("ATOM").
ImapCompiler searches for unallowed symbols in uid-sequence, got ":" and quotes the argument:

case 'ATOM':
case 'SECTION':
    val = (node.value || '').toString('binary');

    if (node.value === '' || imapFormalSyntax.verify(val.charAt(0) === '\\' ? val.substr(1) : val, imapFormalSyntax['ATOM-CHAR']()) >= 0) {
        val = JSON.stringify(val); // HERE
    }

Probable solution

We can implement a new type "uid sequence" or extend allowed symbols. I'm not sure which one solution is preferable.

How to wait and get mail ?

Hello,

I'm currently trying to wait and get new mail then return the parsed source but I don't understand how to use client.idle() or client.noof().
There is no examples for these methods, that's too bad. 😞
Can someone explain me how to use these methods ?
Here my code :
https://gist.github.com/AxelConceicao/b724ee674fa6338d7958abeddad883d3

I'm so sorry to post here, I know it's not the right place but idk where can I get help.

Thanks!

Example of Parsing email

How to Parse my email

Lets say you want to parse your downloaded email

Notice that I use null because I am trying to download the whole content

const { meta, content } = await this.client.download(emailId, null, {
   uid: true,
});

Per documentation content is a stream of a full rfc822 formated message.

So you can use a mailparser like Nodemailers mailparser to parse the attachments of your content.

// get simpleparser like this
const simpleParser = require('mailparser').simpleParser;

// or if using Typescript like I do
import { simpleParser } from 'mailparser';

// Then download the whole email by using null
const { meta, content } = await this.client.download(emailId, null, {
   uid: true,
});

// and simpleParser is able to process the stream as a Promise 
const parsed = await simpleParser(content);

Hope this helps anyone

Disable logger option

Is your feature request related to a problem? Please describe.
Currently there is no clear option to disable logger completely.

Describe the solution you'd like
Disable logger.

Describe alternatives you've considered
Disable logging with option.

const client = new ImapFlow({ logger: false })

In Express, despite providing a custom logger the log also goes to the console

I am testing connection from NodeJS Express code and use suggested NillLogger:

app.post('/comm/test', async (req,res) => {
 class NilLogger { debug(obj) {}; info(obj) {}; warn(obj) {}; error(obj) {} };
 let resp = 'OK', IMAP = new ImapFlow({ logger: new NilLogger(), host: ..., port: ..., secure: true, auth: { user: ..., pass: ... } });
 try {	await IMAP.connect(); } catch (err) { resp = err.response; }
 await IMAP.logout(); res.send({resp})
})

A regular script does't log to the console, but Express post does.

Is their any format for Date in searchObject value "since" and "before"

I am trying to retrieve the emails from a specific time , so i am passing before and after in the searchObject. But its not retrieving any emails. It works for seen=true but it does not return any email or gives any error when using since or before.

let options;

            if (
              req.body.startDate !== undefined &&
              req.body.startDate !== null &&
              req.body.endDate !== undefined &&
              req.body.endDate !== undefined
            ) {
              options = {
                since: startDate,
                before: endDate,
                all: true,
              };
            } else if (
              req.body.startDate !== undefined &&
              req.body.startDate !== null
            ) {
              options = {
                since: startDate,
                all: true,
              };
            } else if (
              req.body.endDate !== undefined &&
              req.body.endDate !== null
            ) {
              options = {
                before: endDate,
                all: true,
              };
            } else {
              options = {
                seen: true,
              };
            }

            for await (let message of client.fetch(options, {
              source: true,
            })) {
              let parsed = await simpleParser(message.source);

              msgs.push({
                // envelope: message,
                myEmail: element.email,
                subject: parsed.subject,
                from: parsed.from,
                body: parsed.text,
                textAsHtml: parsed.textAsHtml,
                date: parsed.date,
              });
            

Unable to download file using download() method

Unable to download file using download() method
I'm trying to download a file using the download() method, but so far I was unable to do it correctly.

To Reproduce
I retrieved an e-mail containing a pdf, and from there i'm trying to get the file

let message = await client.fetchOne(sequenceString, { bodyStructure: true });
      const structuresArray = flattenBodyStructure(message.bodyStructure);
      for (const structure of structuresArray) {
        if (structure.type === 'application/pdf'){
          console.log(structure);
          let {meta, content} = await client.download(sequenceString, structure.part);

          let writeStream = fs.createWriteStream(path.resolve(__dirname, `../downloads/${Date.now()}-${meta.filename}`));

          writeStream.write(content, meta.encoding);

          writeStream.on('finish', () => {
              console.log('wrote all data to file');
          });

          writeStream.end();
        }
      }

flattenBodyStructure is just a convenience method to retrieve the different email parts

Expected behavior
I expect to be able to download the pdf and open it
But the file is corrupt. It seems I do not understand the documentation correctly for the download method.

Empty message after connection closed

Hi,
I create a custom class and when i want to return the variable "message", it's "undefined".

const { ImapFlow } = require('imapflow')

module.exports = class MongoClient {
    constructor() {
        this.client = new ImapFlow({
            host: 'XXX',
            port: 993,
            secure: true,
            auth: {
                user: 'XXX',
                pass: 'XXX'
            }
        })
    }

    async getLastMessage() {
        await this.client.connect()

        let lock = await this.client.getMailboxLock('INBOX')

        let message

        try {
            message = await this.client.fetchOne('*', { source: true })
        } finally {
            lock.release()
        }

        await this.client.logout()

        return message
    }
}

Can you help me please ?
Regards

After clean install

Uncaught TypeError: Invalid Version: undefined
at new SemVer (webpack-internal:///./node_modules/utf7/node_modules/semver/semver.js:279)
at compare (webpack-internal:///./node_modules/utf7/node_modules/semver/semver.js:566)
at Function.gte (webpack-internal:///./node_modules/utf7/node_modules/semver/semver.js:615)
at Object.eval (webpack-internal:///./node_modules/utf7/utf7.js:4)
at eval (webpack-internal:///./node_modules/utf7/utf7.js:121)
at Object../node_modules/utf7/utf7.js (chunk-vendors.js:5817)
at webpack_require (app.js:790)
at fn (app.js:151)
at Object.eval (webpack-internal:///./node_modules/imapflow/lib/tools.js:6)
at eval (webpack-internal:///./node_modules/imapflow/lib/tools.js:699)

Unable to change, add or remove flags

Hi,
Any combination of the following just doesn't work as well as messageFlagsRemove.
uid is correct because I'm using search variable to fetch() the email.

    let search = { uid: 7 };
    let seen = [
        'Seen', '\Seen', '\\Seen', '\\\Seen', '\\\\Seen', 'seen'
    ];
    await client.messageFlagsSet(search, seen);
    await client.messageFlagsAdd(search, seen);

ECONNRESET

sometimes i get this error when connecting

Error: read ECONNRESET
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:111:27)
Emitted 'error' event at:
    at ImapFlow.emitError (C:\Users\User\WebstormProjects\tmail\node_modules\imapflow\lib\imap-flow.js:257:14)
    at TLSSocket._socketError._socketError.err (C:\Users\User\WebstormProjects\tmail\node_modules\imapflow\lib\imap-flow.js:491:22)
    at TLSSocket.emit (events.js:203:15)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)```

Large memory issue when appending email to a mailbox

Hi,

I am using mailcomposer to create a mail buffer with compile().build() to feed it into append function of imapflow and it takes a lot of memory.
I think whether imapflow append could support Stream input or not? Is this possible then I will try to implement it to lessen the memory taken when creating the full message buffer.

Thank you!

`download` hangs

Hi guys, sorry that it's just a question, not a bug of a feature request. Not quite sure what I'm doing wrong below

client.on('exists', async data => {
    for await (let msg of client.fetch('1:*', {bodyStructure: true, envelope: true})) {
        console.log('processing msg', msg.uid, 'from', msg.envelope.from)

        console.log('structure', msg.bodyStructure)

        if (msg.bodyStructure.childNodes)
            await Promise.all(msg.bodyStructure.childNodes.map(async node => {
                if (node.type === 'text/plain') {
                    console.log('downloading part', node.part, 'of message', msg.uid)
                    let {meta, content} = await client.download(msg.uid, node.part)
                    console.log('downloaded part', node.part, 'of message', msg.uid)
                    console.log('meta', meta, 'content', content)
                } else
                    Object.keys(node.parameters).forEach(key => {
                        console.log(key, node.parameters[key])
                    })
            }))
    }
    await client.messageDelete('1:*', {uid: true})
})

the problem is that I never get to "downloaded" console log and it stuck awaiting. Is there something wrong I do?

Thank you in advance for your answer and kudos for your work

download for sendmail IMAP server not working

Can imapflow download attachements from sendmail (dovecot)'s IMAP server?

My example below. the process hangs at the 'if (content)' line and evetually after 30 seconds our test (via tap) timesout.

let unseenCount = await client.status('INBOX',{messages: true, uidValidity: true, unseen: true })
        logger.info('unseenCount returned ', unseenCount)
        if (unseenCount.messages > 0) {
            logger.info('unseenCount returned ' + unseenCount.messages + ' messages')
            for await (let message of client.fetch('1:*',
                {uid: true, envelope: true, bodyStructure: true})) {
                const uid = message.uid
                logger.info(`fetchMany returned UID ${uid} : ${message.envelope.subject}`);
                console.log('body structure ', message.bodyStructure.childNodes)
                let {meta, content} = await client.download('' + uid, '1', {uid: true});
                if (content) {
                    console.log('body part meta ', meta)
                }
            }
     }

Array with SearchObjects fails with exactly 5 SearchObjects

Describe the bug
I fetch emails based on 5 SearchObjects but the query fails. Less than 5 and more than 5 are working though.

The error: 1E BAD Error in IMAP command UID SEARCH: Missing argument (0.001 + 0.000 secs).
The query from ImapFlow: 1E UID SEARCH OR OR OR TO [email protected] TO [email protected] OR TO [email protected] TO [email protected] OR TO [email protected]

To Reproduce

const client = new ImapFlow({ /* config here */ });
const SERACH_OBJS: Object[]  = [
  { to: '[email protected]' },
  { to: '[email protected]' },
  { to: '[email protected]' },
  { to: '[email protected]' },
  { to: '[email protected]' },
]

async function fetchMails() {
  await client.connect();
  let lock = await client.getMailboxLock("INBOX");
  try {
    for await (let message of client.fetch(
        { or: SERACH_OBJS }, /* pass SEARCH_OBS here */
        { envelope: true, uid: true, emailId: true }, /* Another bug: emailID is not fetched! */
        { uid: true }
    )) {
      /* DO STUFF HERE */
    }
  } finally {
    lock.release();
  }
 await client.logout();
}

Expected behavior
Return FetchMessageObject

Logout-command makes NodeJS-process to exit after connection troubles

Description

If a command was interrupted by a manual timeout (as described in #27), the process got "killed" when you try to run logout-command.

To Reproduce

Code:

await client.getMailboxLock('INBOX');

Turn off the internet while a command is running. The client will emit an event-error, but the runnining command will never finished. We can interrupt the running in user-code as described in #27, but NodeJS-process will be suddenly stopped:

await _client.logout(); // HERE

Expected behavior

• Running command should always give control back (by returning or throwing an exception). Command-timeouts should be set too
• Logout-command should handle interrupted state of the client correctly

Environment

• OS: Linux, Ubuntu 20.04
• NodeJS: v14.9.0
• imapflow: v1.0.48

Probable reason

Logout-command sees a currently running command (even when it was "interrupted") and something goes wrong...

Temporary solution in user-code

Don't run logout if any error occurred. Use client.close() directly, clear all resources and create a new client when something goes wrong.

Fix: preCheckQueue is not defined in lib/commands/idle.js

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch [email protected] for the project I'm working on.

preCheckQueue is defined inside a try block but referenced in the catch block, so it triggers a preCheckQueue is not defined Uncaught Exception
preCheckQueue definition should be moved outside the try block

Here is the diff that solved my problem:

diff --git a/node_modules/imapflow/lib/commands/idle.js b/node_modules/imapflow/lib/commands/idle.js
index c06f20d..35d1177 100644
--- a/node_modules/imapflow/lib/commands/idle.js
+++ b/node_modules/imapflow/lib/commands/idle.js
@@ -12,6 +12,8 @@ module.exports = async connection => {
     if (connection.capabilities.has('IDLE')) {
         let response;
 
+        let preCheckWaitQueue = [];
+
         try {
             connection.idling = true;
 
@@ -20,8 +22,6 @@ module.exports = async connection => {
             let doneSent = false;
             let canEnd = false;
 
-            let preCheckWaitQueue = [];
-
             let preCheck = () => {
                 doneRequested = true;
                 if (canEnd && !doneSent) {

This issue body was partially generated by patch-package.

More permissive license?

Would you consider changing the license to something more permissive like MIT (which is used by nodemailer) or BSD or Apache or at least dual-licensing it (like Mozilla does)? AGPL is a no-go for most commercial projects.

"Unexpected Token" with Vue (Webpack)

Describe the bug
I tried to use ImapFlow as shown in the basic example (in the README) with vue.js. When I try to compile the app, I am getting the error you can see below.
It seems like static is not supported in this context, doesn't it?

I have tried to find solutions to this problem, but all solutions are correlated with React (where you can find a webpack config file, which I have not found in vue yet...).

To Reproduce
Steps to reproduce the behavior:

1. Create a new vue app
2. Add a file called MailService in src/services directory.
3. Paste the code from the example. (add a function around it which you module.export =)
4. Import this MailService file into a component and call it.
5. Run npm run serve or npm run build.

Error

 ERROR  Failed to compile with 1 error                                        4:15:37 PM

 error  in ./node_modules/imapflow/lib/imap-flow.js

Module parse failed: Unexpected token (103:19)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
|      * @static
|      */
>     static version = packageInfo.version;
| 
|     /**

 @ ./src/services/MailService.js 1:21-40
 @ ./node_modules/cache-loader/dist/cjs.js??ref--0-0!./node_modules/vue-loader-v16/dist??ref--0-1!./src/components/Mailbox.vue?vue&type=script&lang=js
 @ ./src/components/Mailbox.vue?vue&type=script&lang=js
 @ ./src/components/Mailbox.vue
 @ ./node_modules/cache-loader/dist/cjs.js??ref--0-0!./node_modules/vue-loader-v16/dist??ref--0-1!./src/App.vue?vue&type=script&lang=js
 @ ./src/App.vue?vue&type=script&lang=js
 @ ./src/App.vue
 @ ./src/main.js
 @ multi ./src/main.js

Handler for "exists" Event sometimes not firing

Describe the bug
I open a mailbox and check if new mails are coming in and if so I process them. I achieve this by using imapflow's "exists" event and attaching a function to it. I was not able to figure out the exact circumstances where the handler is not triggered, but sometimes it just doesn't. Possibilities are I am doing something wrong, because I am pretty new to NodeJS.

To Reproduce
Steps to reproduce the behavior:

1. Configuring an event handler for the exists event like so:

const imapClient = new ImapFlow({
    host: process.env.MAIL_HOST,
    port: process.env.IMAP_PORT || "993",
    secure: process.env.IMAP_SECURE || "true",
    auth: {
        user: process.env.MAIL_USER,
        pass: process.env.MAIL_PASSWORD
    }
});

imapClient.on('exists', async () => {
    console.log("EXISTS HANDLER FIRED!");
}

const openMailbox = async () => {
    // open mailbox and get lock
    await imapClient.connect();
    let lock = await imapClient.getMailboxLock("INBOX");
...

2. Sending mails (I send mails with different kinds of attachments, mostly without text content)
3. Most of the times the event is triggered and the function is executed, but sometimes not.

Expected behavior
The function should execute on every "exists" event.

Logs
I added a console.log("EXISTS HANDLER FIRED!") statement at the beginning of the function so it is easy to see in the logs where the function executed. As you can see the mailbox is open and imapflow is idling. Then a mail arrives and we see the EXISTS log message, but the function is not executed. A little bit later another mail comes in, the EXISTS event is fired AND the function is successfully executed.

{"level":20,"time":1585740453423,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"s","msg":"32 IDLE","cid":"weq2gwg6z2h0t2pl0als","v":1}
{"level":20,"time":1585740453466,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"s","msg":"+ idling","cid":"weq2gwg6z2h0t2pl0als","v":1}
{"level":20,"time":1585740453466,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"c","msg":"initiated IDLE","v":1}

{"level":20,"time":1585740481470,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"s","msg":"* 1 EXISTS","cid":"weq2gwg6z2h0t2pl0als","v":1}
{"level":20,"time":1585740481471,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"s","msg":"* 1 RECENT","cid":"weq2gwg6z2h0t2pl0als","v":1}

{"level":20,"time":1585740526757,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"s","msg":"* 2 EXISTS","cid":"weq2gwg6z2h0t2pl0als","v":1}
EXISTS HANDLER FIRED!
{"level":20,"time":1585740526758,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"c","msg":"breaking IDLE","v":1}
{"level":20,"time":1585740526795,"pid":26923,"hostname":"corp","component":"imap-connection","cid":"weq2gwg6z2h0t2pl0als","src":"s","msg":"32 OK Idle completed (0.003 + 73.331 + 73.335 secs).","cid":"weq2gwg6z2h0t2pl0als","v":1}

Unable to receive messages in realtime

Describe the bug
When pooling email inbox using imapflow and new email arrives there, imapflow seems to be unable of receiving it.

To Reproduce
Steps to reproduce the behavior:

1. Run script from this gist against an empty mailbox: https://gist.github.com/georgyfarniev/8c9893a6f5906119bb53753978b6d976
2. Manually send email to this mailbox from another email
3. Observe only undefined value for message.uid in console (since it false, which means nothing was received, I assume)

Expected behavior
imapflow should be able to fetch new message

Desktop (please complete the following information):

• OS: Windwos 10
• Runtime: NodeJS v12.16.3
• Version of imapflow: 1.0.51

Additional context
I double checked this issue, and I can see newly arrived email via thunderbird, but imapflow shows nothing in runtime. If I restart script, it will show message, so problem occurs only if message arrives during runtime. If something is wrong with code, please give me any hint how to solve this issue. Thanks for working on imapflow!

How to get TS typings?

Hello!

In the readme file, you are talking about typing, but I did not find it.

ImapFlow has TS typings set for compatible editors.

Can I get typings?

Thank you!

How can I get only unread emails?

Good morning, I am using this library at work and I have a problem.

When I connect to the imapflow mail server it returns all the mails but I can't differentiate which ones are read and which ones are not read.

I have been reading the documentation (https://imapflow.com/) but I can't find anything that helped me. I have also tried using filters but nothing works.

Could you please help me?

I leave the code snippet where I am trying to get the unread emails.

Thank you very much for your help.

`
const main = async (client) => {

let envelope = [];

await client.connect();

// At this point I have set the variable unseen to true but it doesn't work :(
let lock = await client.getMailboxLock('INBOX', {unseen: true});

try {
    let message = await client.fetchOne('*', { source: true });
    //console.log(message.source.toString());

    for await (let message of client.fetch('1:*', { envelope: true })) {
        
        envelope.push(message.envelope)
    }
} finally {
    lock.release();
}

await client.logout();

let reverse_envelope = envelope.reverse();

return reverse_envelope

};
`

ImapFlow.fetch() returns void in types.d.ts but should return Promise<FetchMessageObject>

Describe the bug
ImapFlow.fetch() returns void in types.d.ts but should return iterable Promise<FetchMessageObject> as described in the imap-flow.js file.

Meanwhile ImapFlow.fetchOne() returns Promise<FetchMessageObject>.

To Reproduce

Steps to reproduce the behavior:
import { ImapFlow } from "imapflow";

const imap = new ImapFlow({...});
await imap.connect();
const lock = await imap.getMailboxLock('INBOX');
try {
  imap.fetch(...); // What type is returned???
} finally {
  lock.release();
}
await imap.logout();

Expected behavior
ImapFlow.fetch() shoudl return iterable Promise<FetchMessageObject>.

Additional context
[email protected]

Incorect Types for messageMove method return object

Hello
The @types/imapflow has a bug, where the return from the messageMove method sais it is boolean, instead of CopyResponseObject which is the correct one.

Imaflow crashes with unhandled error when trying to connect using same client twice

Describe the bug
The program crashes when trying to connect a second time using the same client.

To Reproduce
Steps to reproduce the behavior:

1. connect
2. logout
3. connect again
4. See error

Additional context

...
{"level":20,"time":1615471050885,"pid":12546,"hostname":"pcname","component":"imap-connection","cid":"sal0zyhx49p8x66bu6fd","src":"s","msg":"6 OK Logout completed (0.001 + 0.000 secs).","cid":"sal0zyhx49p8x66bu6fd"}
{"level":30,"time":1615471050915,"pid":12546,"hostname":"pcname","component":"imap-connection","cid":"sal0zyhx49p8x66bu6fd","src":"connection","msg":"Established secure TCP connection","cid":"sal0zyhx49p8x66bu6fd","secure":true,"host":"server1.example.com","servername":"server1.example.com","port":993,"address":"<server ip>","localAddress":"<my ip>","localPort":36184,"authorized":true,"algo":"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256","version":"TLSv1.2"}
{"level":50,"time":1615471050919,"pid":12546,"hostname":"pcname","component":"imap-connection","cid":"sal0zyhx49p8x66bu6fd","err":{"type":"NodeError","message":"write after end","stack":"Error [ERR_STREAM_WRITE_AFTER_END]: write after end\n    at writeAfterEnd (_stream_writable.js:266:14)\n    at ImapStream.Writable.write (_stream_writable.js:315:5)\n    at TLSSocket.ondata (_stream_readable.js:718:22)\n    at TLSSocket.emit (events.js:314:20)\n    at addChunk (_stream_readable.js:297:12)\n    at readableAddChunk (_stream_readable.js:272:9)\n    at TLSSocket.Readable.push (_stream_readable.js:213:10)\n    at TLSWrap.onStreamRead (internal/stream_base_commons.js:188:23)","code":"ERR_STREAM_WRITE_AFTER_END"},"cid":"sal0zyhx49p8x66bu6fd"}
events.js:291
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:266:14)
    at ImapStream.Writable.write (_stream_writable.js:315:5)
    at TLSSocket.ondata (_stream_readable.js:718:22)
    at TLSSocket.emit (events.js:314:20)
    at addChunk (_stream_readable.js:297:12)
    at readableAddChunk (_stream_readable.js:272:9)
    at TLSSocket.Readable.push (_stream_readable.js:213:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:188:23)
Emitted 'error' event on ImapFlow instance at:
    at ImapFlow.emitError (/home/username/DevProjects/WebstormProjects/projectname/node_modules/.pnpm/[email protected]/node_modules/imapflow/lib/imap-flow.js:257:14)
    at ImapStream.<anonymous> (/home/username/DevProjects/WebstormProjects/projectname/node_modules/.pnpm/[email protected]/node_modules/imapflow/lib/imap-flow.js:252:18)
    at ImapStream.emit (events.js:326:22)
    at errorOrDestroy (internal/streams/destroy.js:108:12)
    at writeAfterEnd (_stream_writable.js:268:3)
    at ImapStream.Writable.write (_stream_writable.js:315:5)
    [... lines matching original stack trace ...]
    at readableAddChunk (_stream_readable.js:272:9) {
  code: 'ERR_STREAM_WRITE_AFTER_END'
}

Sample code:

const client = new ImapFlow({
    host: keys.mailHost,
    port: keys.mailPort,
    secure: true,
    auth: {
        user: keys.mailUser,
        pass: keys.mailPass
    }
});

(async function() {
    await client.connect();
    await client.logout();
    await client.connect();
}())

Gmail and Application Password failing to open mailbox

Describe the bug
Imapflow is failing to open a mailbox in a gmail account with Application Password login.
The client is passing the connect stage check but failing to open a mailbox, it returns Command failed error with no more info about whats going on than that.

To Reproduce
Steps to reproduce the behavior:

1. instantiate Imapflow with gmail and Application Password settings
2. Connect with await client.connect()
3. Open any mailbox, eg: client.mailboxOpen("INBOX");
4. Throws a Command failed error.

Expected behavior
Open the mailbox as normal or a detailed error message

Desktop (please complete the following information):

• OS: Fedora 34 Linux
• NodeJS: 16.9.0

Can not import FetchMessageObject type from imapflow

Describe the bug
FetchMessageObject is located in the file types.d.ts but it's out of the module scope therefore it's impossible to import as:
import { ImapFlow, FetchMessageObject } from "imapflow";

To Reproduce
Steps to reproduce the behavior:

1. npm install imapflow
2. import { ImapFlow, FetchMessageObject } from "imapflow";

Expected behavior
FetchMessageObject type should be exported along with the module.

Additional context
[email protected]

Extend german translations for special use

Is your feature request related to a problem? Please describe.

Recently I wanted to use imapflow to handle an email account with the following host: outlook.office365.com.
Unfortunately the boxes don't contain the capability XLIST or SPECIAL-USE therefore the following string comparison happens in special-use.js:

let name = folder.name.toLowerCase().trim();
return Object.keys(module.exports.names).find(flag => module.exports.names[flag].includes(name));

In this case the folders german names differ slightly from what is currently available and therefore the specialUseFlag is missing for several boxes.
I am not able to rename boxes.

Describe the solution you'd like

Since the folders german names aren't anything special, but more general terms, I'd like to add those to the corresponding arrays in special-use.js.

Additional context

I'd like to create a PR, but unfortunately I cannot create branches. How can I contribute?

Typescript types not picked up by tsc

While types are included in the npm package, the types field in package.json is not set, resulting in Typescript not picking up types for this library.