jhermsmeier/node-envelope

Quite liberal Email parser

Envelope

Envelope parses emails quite liberally into an object structure which makes it easy to work with.

Install via npm

$ npm install --save envelope

Features

• Parses almost everything. If it doesn't: file an issue
• Decodes MIME words, base64, ...
• Automatically converts to UTF8 w/ iconv (codes)
• Converts attachments to buffers

Performance

On an Intel Core i5-3427U CPU @ 1.80GHz it processes about 20.1 MB per second, which equals roughly 215.8 mails per second.

Usage

Parsing an email

var fs = require( 'fs' )
var Envelope = require( 'envelope' )

// Read email into a buffer
var data = fs.readFileSync( './test.eml' )

// Construct envelope
var email = new Envelope( data )

console.log( email )

Example Output:

{
  header: {
    received: [
      'by mail-wi0-f175.google.com with SMTP id hm11so5717280wib.2 for <[email protected]>; Sat, 22 Dec 2012 07:49:06 -0800 (PST)',
      'by 10.194.78.162 with SMTP id c2mr28698959wjx.46.1356191346691; Sat, 22 Dec 2012 07:49:06 -0800 (PST)',
      'by 10.194.64.229 with HTTP; Sat, 22 Dec 2012 07:49:06 -0800 (PST)'
    ],
    dkimSignature: 'v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=DrlXO8ocnosZnW5ZN7P4S/fIdR8vwHj0TyzoPISZF2Q=; b=gOHBExs2JcJFRrozPDw88Js0dc0AHOo6YTZqrDTedfcK/jM/mxfu5rfVzuUnKAGiS5 ZvRvXvwYjIW0B9t0DDHDOs5soIukuEXeUw9OV2QD8qc5pmOShuRQWyW5pRftTF87omkj gV2Eik5K2f8FpNlyvuLDjMUmyP8RpLaRrii6+kRRsoJzzP41IqALmlLmJfvtnkeu5kM0 v4XnQ4hBNcaLuCmq3fZfCQFDexofECQOZ8FWE0VfdASG8HOJ6jgxuKwYtNfy11ySUSrI wFFlrjTfiNqSD9nzQns3j+xXLtqsvviJQXJgkC8O6mLel3GDwm8LHzBoszzqZ/FiL4rg Vdfw==',
    mimeVersion: '1.0',
    date: 'Sat, 22 Dec 2012 16:49:06 +0100',
    messageId: '<CA+0p7-rrsAij-6nzDgk3R62ZHRZrjdJvOjxhCsHQ+m=[e-mail]>',
    subject: 'AGAIN',
    from: {
      address: '[email protected]',
      name: 'Jonas Hermsmeier'
    },
    to: {
      address: '[email protected]',
      name: null
    },
    contentType: {
      mime: 'multipart/alternative',
      boundary: '047d7bfd046e778e8d04d172e7cb'
    }
  },
  '0': {
    '0': 'HELO',
    header: {
      contentType: {
        mime: 'text/plain',
        charset: 'UTF-8'
      }
    }
  },
  '1': {
    '0': '<div dir=\\"ltr\\">HELO</div>',
    header: {
      contentType: {
        mime: 'text/html',
        charset: 'UTF-8'
      }
    }
  }
}

Using filters

Transforming header field values

You can easily add your own transformation functions to Envelope. To transform a specific header field value, just make sure the field identifiers are written in camelCase:

Envelope.Header.filter.add(
  'dkimSignature', function dkim( value ) {
    // ...
    return newValue
  }
)

If you want to apply your transformation to a set of header fields, simply use an array of field names, e.g. instead of 'contentType' use [ 'contentType', 'contentDisposition' ].

API

new Envelope( buffer )

Contructs a new envelope object from a buffer.

node-envelope's Issues

Typescript definition

Would be really nice to have a typescript definition for this.

Is there some documentation from where I can copy all of the definitions?

BTW: Amazing library you've created, I've had a lot of use for it :)

mails with format ASCII only LF line endings are not parsed

It looks like only mails with CRLF line endings are parsed. I will look into contributing a patch to fix that.

Update & improve documentation

It's lacking. A lot. ref #19

Add a plugin system

So one can, for example, validate signatures (DKIM, PKCS, ...)
Concept:

var Envelope = require( 'envelope' )
var DKIM = require( 'envelope-dkim' )

Envelope.use( DKIM({
  algorithm: 'rsa-sha256'
}))

var email = new Envelope( data )
email.dkim.sign()

erroneous subject field parsing (and possibly other fields to?)

raw email:

Subject: =?utf-8?B?QVc6IEdyb8OfZSBHZWJ1cnRzdGFncy1QYXJ0eSBpbSBN?=
	=?utf-8?B?YWkgKGljaCBmZWllcmUgbmFjaHRyw6RnbGljaCkh?=

Output:
AW: Große Geburtstags-Party im M ai (ich feiere nachträglich)!

Should Be:
AW: Große Geburtstags-Party im Mai (ich feiere nachträglich)!

the field "threadTopic" shows the space at a different place int he text, so it is also affected.

Parsing quote-printable body content problem

Hello.

I have the following body content which has line feeds but the envelope part corresponding to the text/plain body part came without line feeds.

------=_Part_1796134_734941703.1461597339659
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

=C2=A0
Get Firefox!

     On Thursday, April 21, 2016 8:19 PM, zzzz zzz <[email protected]> wr=
ote:
=20

 =C2=A0
Get Firefox!

I need to display this body content in an webmail like page but the space between "Get Firefox! and On..." is lost

As far I understand this happened in:

[email protected]:22
[email protected]:52
[email protected]:148

Implement header field validation

Implement header field transforms

Implement a streaming parser

Add tests

buffer slice is out of bounds

Example:

Received: (qmail 59857 invoked by uid 2); 6 Dec 2012 13:14:47 -0000
Date: 6 Dec 2012 13:14:47 -0000
Message-ID: <[email protected]>
From: [email protected]
To: [email protected]
Subject: new pw

 New Hacker News pw: XXXXXX

Fails with sourceStart out of bounds in Buffer.prototype.copy()

message body not decoded when contentTransferEncoding is uppercase

The contentTransferEncoding header indicates weather the body is encoded as "quoted-printable". Some messages have a header with uppercase "QUOTED-PRINTABLE" and these are not decoded by envelope.

Error: missing header

I get a "Missing Header" error when parsing any message on my server. Here's an example:

Return-Path: <[email protected]>
Delivered-To: [email protected]
Received: from mail.kuba-orlik.name
	by mail.kuba-orlik.name with LMTP
	id 8JzJHNlyhF6ERwAApHATFQ
	(envelope-from <[email protected]>)
	for <[email protected]>; Wed, 01 Apr 2020 10:54:17 +0000
Received: from localhost (localhost [127.0.0.1])
	by mail.kuba-orlik.name (Postfix) with ESMTP id 6C671107335
	for <[email protected]>; Wed,  1 Apr 2020 10:54:17 +0000 (UTC)
X-Virus-Scanned: Yes
To: [email protected]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kuba-orlik.name;
	s=mail; t=1585738456;
	bh=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=; h=To:From:Subject;
	b=f1HMaLuFaOQ4muP1hf96BJ8znuR1LcCkdVKk6qLfkZ89Q7h81Y5D3tBUQ58DX9blB
	 RXOscAcVzdYYBFHOHiz2uCMFs0f/SMZZcXLJMDTZXJtmdHGuYYeSHTTHsnDguhpweb
	 EtEcnp0d8m0DTVlbWlr8qF5RW/Vqc3dTIck5KVJBzKyVp9Ty+JhMNPH+vJxo33hQFJ
	 9nLJmeqpFp3CayvooJJnuNs+gfcQhZ8zVO0rl2/KGAmpH0BHGAaft3gb5fhNvENPck
	 evvWyUcT/OTxzMXjDDLdoBaYQNhNT1NpKm7lMNTYFXJHYxMKrD4EcfloaoU2t+cf5R
	 5YruOoXMNomgg==
From: Kuba Orlik <[email protected]>
Subject: iaofjodifajoidfjoiad90
Message-ID: <[email protected]>
Date: Wed, 1 Apr 2020 12:54:15 +0200
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US

envelope doesn't parse more than 1 attachment

Emails sent from gmail with multiple attachments don't get parsed fully, only 1 of the attachments is "parsed"

Steps to reproduce

1. Send an email with multiple attachments
2. Parse it on the server

Code used

const envelope = require("envelope");
const fs = require("fs");

let parsed = envelope.parse(fs.readFileSync("./raw.txt"));

setInterval(() => {}, 1e6);

if (process.platform === "win32") {
    let r = require("readline").createInterface({
        input: process.stdin,
        output: process.stdout
    });

    r.on("line", line => {
        if (!line) return;

        try {
            let res = eval(line);
            console.log(res);
        }
        catch (ex) {console.log(ex);}        
    });
}
//allows easy inspection of the parsed email by just eval-ing stdin

Output

I can provide the "raw email" file if needed

erroneous parsing of from field

raw email:
From: "Almaifd, Tim" <[email protected]>

return:

from:
   [ { address: '"Almaifd', name: null },
     { address: '[email protected]', name: ' Tim"' } ],

should be:

from: { address: '[email protected]', name: 'Almaifd, Tim' },

Browser support

Your dependencies all appear that they would work in the browser, at least when pulled in through Rollup + rollup-plugin-node-builtins. Would you be willing to have source converted to ESM (producing also a UMD build)?

Encoding error

Hi,

While running node-envelope against a long list of EMLs exported from Outlook, I got a crash on a particular email.

Error: Incomplete character sequence.
    at errnoException (/Users/rprieto/Documents/Dev/rename-eml/node_modules/envelope/node_modules/iconv/lib/iconv.js:160:13)
    at Object.convert (/Users/rprieto/Documents/Dev/rename-eml/node_modules/envelope/node_modules/iconv/lib/iconv.js:136:17)
    at Iconv.convert (/Users/rprieto/Documents/Dev/rename-eml/node_modules/envelope/node_modules/iconv/lib/iconv.js:54:12)
    at Object.MIME.decodeQP (/Users/rprieto/Documents/Dev/rename-eml/node_modules/envelope/node_modules/mime-lib/mime.js:157:8)
    at Body (/Users/rprieto/Documents/Dev/rename-eml/node_modules/envelope/body/index.js:48:24)
    at Envelope (/Users/rprieto/Documents/Dev/rename-eml/node_modules/envelope/envelope.js:23:17)

I reduced that particular EML to the simplest single-line content that sill crashes:

From: <[email protected]>
To: <[email protected]>
Subject: Subject line
Date: Tue, 3 Jun 2003 19:18:06 +0200
MIME-Version: 1.0
Content-Type: text/plain;
    charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Bonne soir=E9e

The email body seems to be the issue. It's the French sentence Bonne soirée, and I suppose the encoding for é is correct, given the EML is straight out of Outlook.

Any pointers to troubleshoot this?
Thanks!

TypeScript types

It'd be great to have types for envelope... Thanks!

Implement un/serialization

Implement Envelope#toString()

• Implement header serialization
• Implement body serialization
• Implement attachment serialization

Implement body part transforms

Quoted-printable strings in subject not decoded correctly

Hello,
I just found your library and it seems to be simple and produces really clean objects but I found one issue when parsing Subject headers that are quoted-printable.

My email.eml file contains this line: Subject: =?UTF-8?Q?a=c5=82o=c5=9b-?=

Running it through envelope with this code:

var fs = require( 'fs' )
var Envelope = require( 'envelope' )
var data = fs.readFileSync( './email.eml' )

var email = new Envelope( data )
console.log(email.header.subject);

// libqp for comparison
var libqp = require('libqp');
var s = libqp.decode('a=c5=82o=c5=9b-').toString('utf8');
console.log(s);

Produces the following output:

a=c5�o=c5=9b- // badly decoded string from envelope
ałoś-          // libqp parses this correctly

I've also opened the file in an e-mail client and surely enough the subject was displayed correctly.

I'm using version 2.0.0 (previous version 1.0.9 did not touch quoted-printable subject at all I think).

node-imap

Does this project support direct parsing of node-imap objects? If it does, I will try and figure out how the code works, and add some documentation and examples for future users.