tonioo/sievelib

Client-side Sieve and Managesieve library written in Python.

sievelib

Client-side Sieve and Managesieve library written in Python.

• Sieve : An Email Filtering Language (RFC 5228)
• ManageSieve : A Protocol for Remotely Managing Sieve Scripts (RFC 5804)

Installation

To install sievelib from PyPI:

pip install sievelib

To install sievelib from git:

git clone [email protected]:tonioo/sievelib.git
cd sievelib
python ./setup.py install

Sieve tools

What is supported

Currently, the provided parser supports most of the functionalities described in the RFC. The only exception concerns section 2.4.2.4. Encoding Characters Using "encoded-character" which is not supported.

The following extensions are also supported:

• Copying Without Side Effects (RFC 3894)
• Body (RFC 5173)
• Vacation (RFC 5230)
• Seconds parameter for Vacation (RFC 6131)
• Relational (RFC 5231)
• Imap4flags (RFC 5232)
• Regular expression (Draft)

The following extensions are partially supported:

• Date and Index (RFC 5260)
• Checking Mailbox Status and Accessing Mailbox Metadata (RFC 5490)

Extending the parser

It is possible to extend the parser by adding new supported commands. For example:

import sievelib

class MyCommand(sievelib.commands.ActionCommand):
    args_definition = [
        {"name": "testtag",
            "type": ["tag"],
            "write_tag": True,
            "values": [":testtag"],
            "extra_arg": {"type": "number",
                          "required": False},
            "required": False},
        {"name": "recipients",
            "type": ["string", "stringlist"],
            "required": True}
    ]

sievelib.commands.add_commands(MyCommand)

Basic usage

The parser can either be used from the command-line:

$ cd sievelib
$ python parser.py test.sieve
Syntax OK
$

Or can be used from a python environment (or script/module):

>>> from sievelib.parser import Parser
>>> p = Parser()
>>> p.parse('require ["fileinto"];')
True
>>> p.dump()
require (type: control)
    ["fileinto"]
>>>
>>> p.parse('require ["fileinto"]')
False
>>> p.error
'line 1: parsing error: end of script reached while semicolon expected'
>>>

Simple filters creation

Some high-level classes are provided with the factory module, they make the generation of Sieve rules easier:

>>> from sievelib.factory import FiltersSet
>>> fs = FiltersSet("test")
>>> fs.addfilter("rule1",
...              [("Sender", ":is", "[email protected]"),],
...              [("fileinto", "Toto"),])
>>> fs.tosieve()
require ["fileinto"];

# Filter: rule1
if anyof (header :is "Sender" "[email protected]") {
    fileinto "Toto";
}
>>>

Additional documentation is available within source code.

ManageSieve tools

What is supported

All mandatory commands are supported. The RENAME extension is supported, with a simulated behaviour for server that do not support it.

For the AUTHENTICATE command, supported mechanisms are DIGEST-MD5, PLAIN, LOGIN and OAUTHBEARER.

Basic usage

The ManageSieve client is intended to be used from another python application (there isn't any shell provided):

>>> from sievelib.managesieve import Client
>>> c = Client("server.example.com")
>>> c.connect("user", "password", starttls=False, authmech="DIGEST-MD5")
True
>>> c.listscripts()
("active_script", ["script1", "script2"])
>>> c.setactive("script1")
True
>>> c.havespace("script3", 45)
True
>>>

Additional documentation is available with source code.

sievelib's Issues

FiltersSet - Example not working

The following example in the documentation doesn't work:

#!python

>>> from sievelib.factory import FilterSet
>>> fs.addfilter("rule1",
...              [("Sender", ":is", "[email protected]"),],
...              [("fileinto", "Toto"),])
>>> fs.tosieve()
require ["fileinto"];

# Filter: rule1
if anyof (header :is "Sender" "[email protected]") {
    fileinto "Toto";
}
>>>

First, there's a typo. I found out that "FilterSet" is wrong. Correct is "FiltersSet". But where is the variable fs defined? Could you provide an example that actually runs?

Thanks a lot

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/8
• Originally Reported By: Anonymous
• Originally Created At: 2012-08-30T10:23:34.036

missing vacation command

The vacation command is not implemented

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/1
• Originally Reported By: Anonymous
• Originally Created At: 2011-07-13T10:38:35.334

Bad parsing of multi-line strings

The parsing works well when only one string of this type is present. It fails when more are present: they are all considered as one multi-line string because the regexp is greedy.

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/12
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2013-01-19T17:29:45.134

add_commands documentation

Is there any document for adding command?
I have tried to add command but it gives error

  File "/opt/pyvirtual/MMC-py_3.4-dj_1.9/lib/python3.4/site-packages/sievelib/factory.py", line 200, in __create_filter
    if action.is_extension:
AttributeError: 'NoneType' object has no attribute 'is_extension'

Better support of relational and date extensions

The current implementation is buggy.

Example for vacation filter creation

Hi,

First thank you fr sievelib.
I try to add a filter with factory for a vacation rule, but it takes only the 2 first arguments for the action ("vacation","subject","...etc,..."). I check the commands.py for vacation and the command exist with all arguments.
Please, can you give me a example for a vacation command to add days, subject, text, from, etc...

Thanks in advance.

Improve the reading of server responses

Literals are not handled at all.

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/5
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2012-03-14T16:24:37.721

Issue with content with non-ASCII characters

The python3 compat. introduced a regression with non ASCII characters.

Handling of "not" keyword fails

#!sieve

        if allof (
 not (
 header :contains ["To","Resent-to"] "[email protected]" ) ){
        discard;
                }

For all rules containing the not keyword this parsing error occurs:
parsing error: left_parenthesis found while identifier expected near '
'

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/19
• Originally Reported By: Daniël van Eeden
• Originally Created At: 2013-08-02T13:50:35.172

Sieve date extention support (RFC 5260)

It would be nice to have support for the set and currentdate commands by default.

line 3: unknown command currentdate
line 3: unknown command set

https://tools.ietf.org/html/rfc5260

#!sieve

#RULE: $Name="datetimetest" $Order=1 $Type="TIMED_DEFAULT_TYPE"
        require ["variables","relational","date"];
        set "startDate" "2012-11-01";
        set "endDate" "2013-12-31";
        if allof (
 currentdate :zone "+0100" :value "ge" "date" "${startDate}",
 currentdate :zone "+0100" :value "le" "date" "${endDate}" ){
#BEGINFILTER
        if anyof (
 header :contains ["From","Sender","Resent-from","Resent-sender","Return-path"] "foo",
 header :contains ["From","Sender","Resent-from","Resent-sender","Return-path"] "bar" ){
        redirect "[email protected]";
        keep;
                }
#ENDFILTER

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/18
• Originally Reported By: Daniël van Eeden
• Originally Created At: 2013-08-02T10:07:36.959

Sieve extension: Copying Without Side Effects

Implement this extension into the parser: https://tools.ietf.org/html/rfc3894

Can't connect to sieve server using IPv6 address

Shouldn't be too difficult to fix, I'll create a PR when I get a chance.

sievelib/sievelib/managesieve.py

Line 499 in cd06074

self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

from ipaddress import ip_address

server_ip = ip_address(self.srvaddr)
sock_family = socket.AF_INET6 if server_ip.version = 6 else socket.AF_INET
self.sock = socket.socket(sock_family, socket.SOCK_STREAM) 

requirements.txt

ipaddress; python_version < '3.3'

Installation of latest release fails with current pip versions

The master branch does not have this issue anymore. Is there any ETA for the new release?

Manage other users

Sieveshell with Cyrus-IMAP allows for an "authentication name" (-a) as well as an "authorization name" (-u)
The -a parameter states which user is authenticating to the server. The -u parameter states which user the sieve script will be manage for. This allows for a admin type user to log in and manage other users. This is handy for environments where control to the various servers is necessary.

I believe the -u parameter is implied from the login user if only the -a parameter is specified. It would be grand if similar functionality existed in:

def connect(self, auth, login, password, starttls=False, authmech=None):

Where login is the managing admin and auth is the person to be managed.

Thoughts?

:addresses without a list

Not sure if this is covered by the RFC, but with Roundcube I get a sieve script that contains something like this::addresses "[email protected]"

The parser fails in this case because it should be a list:

>>> p.error
'line 12: bad value "[email protected]" for argument addresses'

So it seems to me that :addresses can either be a list of mail addresses or a string containing a mail address.

Plans for body extension?

Hello,

Are there any plans for supporting the body extension?
https://tools.ietf.org/html/rfc5173

Thanks and keep up the good work!

"redirect" command not known as a Sieve capability

When trying to make use of the "redirect" command via Modoboa, the error message that pops up reads the following: "'redirect' is not known as a Sieve capability, but it is known as a Sieve command that is always available.'

Based on an IRC discussion this issue seems to be linked with sievelib.

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/23
• Originally Reported By:
• Originally Created At: 2013-08-31T07:34:26.596

STARTTLS not recognised

I work with a cyrus Server and if I start making a call to sievelib.managesieve.Client.connect with starttls=True i get the "STARTTLS not supported by the server".

With Wireshark i can see the STARTTLS in the capabilities right after the connect on the last line (before the finishing OK).

When I replace
for l in capabilities.splitlines()[0:-1]:
by
for l in capabilities.splitlines():
in sievelib/managesieve.py on line 260
the connect is working fine.

Factory: add support for currentdate test

Make the factory module understand the currendate test.

some stringlist tags seem to be missing after parsing

Hi Antoine,

i just came across this one.
Consider the following Sieve filter:

#!

require ["vacation"];
if
  address :contains ["From", "Sender"] ["[email protected]"]
  {
vacation :days 5 :addresses ["[email protected]"] "I'm away!";

  }

After parsing and dumping it it becomes:

#!

require ["vacation"];

if address :contains ["From", "Sender"] ["[email protected]"] {
    vacation :days 5 ["[email protected]"] "I'm away!";
}

What is missing is the ":addresses" part in the vacation command. I think it has to do with the stringlist handling but i am not sure... Do you have a clue?

Cheers,
Sebastian

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/14
• Originally Reported By:
• Originally Created At: 2013-02-01T12:08:50.360

Parse to json

How i can parse filters to json?

Managesieve/putscript : Incorrect support of unicode objects

The client sends a wrong literal to the server. It uses the length of the unicode object, not the one of the str object which is finally sent on the socket.

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/4
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2012-03-14T16:23:08.407

Capabilities parsing error.

There's a small bug on the code that can lead to incorrect capabilities's parsing.
Diff

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/7
• Originally Reported By: Gérald Colangelo
• Originally Created At: 2012-06-27T17:22:34.881

How to get conditions and actions lists from FiltersSet?

Hello Tonioo,
looking at the documentation I see it is possible to add a new filter passing a name and conditions and actions list

from sievelib.factory import FiltersSet
fs = FiltersSet("test")
conditions = [("Sender", ":is", "[email protected]"),]
actions = [("fileinto", "Toto"),]
fs.addfilter("rule1", conditions, actions)
fs.tosieve()

What I need is to export the two lists (conditions and actions) from FiltersSet, something like this:

p = Parser()
p.parse('....sieve rule....')
fs = FiltersSet("test")
fs.from_parser_result(p)
# the following lines is what I would need
conditions = fs.get_conditions()
actions = fs.get_actions()

Is it possible to add the two above functions or can you help me understanding how to obtain the two arrays?
Thanks!

empty require when there was no require in the sieve

Hi Antoine,

i have the following sieve:

#!

if address :contains ["From", "Sender"]
  ["[email protected]",
  "[email protected]"]
  { discard; stop; }

Note that it doesn't "require" a module.
The resulting Sieve after passing it to FiltersSet starts with

#!

require [];

so, it's an empty require, which shouldn't be there :)

Cheers,
Sebastian

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/11
• Originally Reported By:
• Originally Created At: 2012-12-14T12:28:27.122

Vacation support patch

Hi,

Please, can you consider adding this patch to sieve lib in order to support vacation command.

As //vacation// is a defacto command used by almost all sieve implementations.

Not having vacation support forces parser to create errors.

Thanks,

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/2
• Originally Reported By: Jordi Llonch
• Originally Created At: 2011-09-10T06:58:45.046

New extension : reject

Add support for the reject extension (RFC 5429)

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/9
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2012-11-21T17:38:08.578

Support negation when using factory

It is not currently possible to construct negative tests (ie. not header :contains ...)

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/24
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2013-12-16T14:44:59.156

Default manage sieve port is not the IANA allocated one.

Please change the default manage sieve port from 2000/tcp to 4190/tcp as listed in the
IANA registry.

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/20
• Originally Reported By: Daniël van Eeden
• Originally Created At: 2013-08-09T11:26:46.537

Better string handling

The factory class should handle string a bit better and automatically quote them so the parser won't consider them a multi-line strings.

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/17
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2013-04-01T09:12:51.283

Factory: add support for body and envelope tests

Those tests are useful and should be handled by the factory module.

Endless loop while server "Connection refused"

EndLess loop
This is the line when imap server refused a connection make endless loop.
If you set timeout for this class. Why not use while time.now - time.begin < timeout instead while True.

Plain authentication doesn't support non-ascii characters

The authentication method mixes str and unicode type.

Support 'exists' test in factory

Add support for 'exists' tests when using the factory.

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/25
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2013-12-16T15:05:49.194

The factory always generates a require statement

A require statement is generated for each filter action even if it is not an extension. (example: the redirect command).

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/22
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2013-08-28T21:48:39.787

Support for CHECKSCRIPT in managesieve

The sievelib.managesieve.Client() lacks a checkscript() function.

http://tools.ietf.org/html/rfc5804#section-2.12

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/21
• Originally Reported By: Daniël van Eeden
• Originally Created At: 2013-08-09T11:35:41.960

left_bracket found while semicolon expected

Hi Antonio,

i tried digging and debugging but i can't figure out what's wrong here..
I want to add a new SIEVE command called quota_notification which looks like:

QUOTA_NOTIFICATION [:subject ] :recipient

So i created:

#!python

class Quota_notificationCommand(ActionCommand):
    args_definition = [
        {"name": "subject",
         "type": ["tag"],
         "write_tag": True,
         "values": [":subject"],
         "extra_arg": {"type": "string"},
         "required": False},
        {"name": "recipient",
         "type": ["tag"],
         "write_tag": True,
         "values": [":recipient"],
         "extra_arg": {"type": "stringlist"},
         "required": True}
    ]

And while trying to parse

#!

quota_notification :subject "subject here" :recipient ["[email protected]"];

The parser stops at the first left bracket. The parser wants the recipient command to be the last and sets the next expected symbol to semicolon. Do you know why?
I'd appreciate a hint! :)

Cheers,
Sebastian

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/15
• Originally Reported By:
• Originally Created At: 2013-02-05T10:29:50.332

Encoding errors

There are still some encoding errors when converting from unicode to string ('ascii' used by default).

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/6
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2012-06-12T09:09:22.137

addfilter doesn't work anymore

The addfilter method is currently broken if you don't specify a description (which is an optional field).

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/16
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2013-04-01T09:09:27.023

Python3 compatibility

Add support for Python >= 3.3

Updating filters without name

Hello,

let us suppose that I want to manage my existing filters with a program which uses sievelib. If I understand correctly, if I wanted to add a new condition to an 'IfCommand' I should:

1. parse my filters
2. find the one that I want to modify
3. construct a new one with the previous conditions and the new condition
4. use updatefilter or replacefilter
5. store the new filter

My problem:
updatefilter needs a name in order to work. Wouldn't it be nice to have the possibility to change a filter by its position instead of the name? Maybe I'm wrong and this is not the adequate way of dealing with these things.

Thanks!

Parsing stops on not anyof

example.sieve.txt

When I parse that file with sievelib 1.1.1, it stops processing (without error) on this conditional:

if not anyof(size :under 1234, size :over 1234) {
    fileinto "Test 9";
}

It does not read or return the remaining commands in the file. E.g.:

Python 3.7.2 (default, Jan  3 2019, 02:55:40)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from sievelib.parser import Parser
>>> p = Parser()
>>> p.parse_file('example.sieve.txt')
True
>>> p.result
[require (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control)]
>>>

Expected behavior would be:

>>> p.result
[require (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), if (type: control), elsif (type: control), elsif (type: control), else (type: control)]

Py3 compat: sock.recv() returns bytes, not str - TypeError in connect() in managesieve.py

I have looked into your mentions of Python3 compatibility, and created a virtualenv with Python 3.6.
As soon as STARTTLS is initialized, the following exception appears when reading from the socket:

  File "~/.virtualenvs/sievelib/lib/python3.6/site-packages/sievelib/managesieve.py", line 489, in connect
    if not self.__get_capabilities():
  File "~/.virtualenvs/sievelib/lib/python3.6/site-packages/sievelib/managesieve.py", line 267, in __get_capabilities
    code, data, capabilities = self.__read_response()
  File "~/.virtualenvs/sievelib/lib/python3.6/site-packages/sievelib/managesieve.py", line 198, in __read_response
    line = self.__read_line()
  File "~/.virtualenvs/sievelib/lib/python3.6/site-packages/sievelib/managesieve.py", line 162, in __read_line
    self.__read_buffer += nval
TypeError: must be str, not bytes

This error appears, because Python 3 returns Bytes in socket.recv(), not str as in 2.7. A concatenation with the initialized self.__read_buffer = "" (str) does not work.

It's easy to reproduce:

>>> a = "abc"
>>> b = 'def'.encode('utf8')
>>> a += b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: must be str, not bytes
>>> type(a), type(b)
(<class 'str'>, <class 'bytes'>)
>>> a += b.decode('utf8')
>>> a
'abcdef'

This can be fixed with self.__read_buffer += nval.decode('utf8') in managesieve.py line 162, but the TypeError then appears at another place. So I guess it's a general socket handling compatibility problem.

stop action is wrongly implemented

It is currently declared as a control command.

Issue with unicode filter names

A python 2/3 compatibility issue still exists with unicode filter names.

new Command implementation

Hi Antoine,

i am wondering,
i came across a proprietary command in my mail server.
I am trying to implement it in commands.py.
The command looks like:

#!

# cp_filter_name: anti viagra
if header :matches "Subject" "*gra"
{
cp_notify :body 160 ["[email protected]"];
}

:body is optional (as well as a :recipients parameter).

I added the following:

#!python

class Cp_notifyCommand(ActionCommand):
    args_definition = [
        {"name" : "recipients",
         "type" : ["string", "stringlist"],
         "required" : False},
        {"name" : "body",
         "type" : "number",
         "required" : False}
        ]

Before the parser complained with "unknown command", now it is:
"Sieve Filter Parsing failed: line 14: parsing error: unexpected token ':body' found near ' '"

Do you have a hint or an idea what is wrong?

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/13
• Originally Reported By:
• Originally Created At: 2013-01-30T16:23:07.566

The parser does not support all multi-line strings

Currently, the implementation does not support strings of the form :

text:
Bla
Bla
.

which is clearly allowed by the RFC:

multi-line         = "text:" *(SP / HTAB) (hash-comment / CRLF)
                     *(multiline-literal / multiline-dotstart)
                     "." CRLF

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/3
• Originally Reported By: Antoine Nguyen
• Originally Created At: 2011-09-12T14:08:52.575

Hashcomments are displaced

Hi Antoine,

i am trying to figure out how to make the sievelib work better with hash comments.
Problem is that my sieve filters can contain 0, 1 or 2 hash comments before them (indicating unknown filters, filters with a name and filters with a name and a description). Also the names are not marked by "Filter name" but some other text.
Now, with the current implementation this all gets mixed up because hash comments are kept seperate.
Any ideas?

Cheers,
Sebastian

---

• Bitbucket: https://bitbucket.org/tonioo/sievelib/issue/10
• Originally Reported By:
• Originally Created At: 2012-12-11T11:33:48.749

Bad error message raised on known response codes

I've got a QUOTA/MAXSIZE response code and it raised a "Bad error message" on sievelib.

Example:
(QUOTA/MAXSIZE) "Script is too large (max 1048576 bytes)."

RFC
https://tools.ietf.org/html/rfc5804#section-1.3

Parsing bug on anyof and allof

this filter

require ["fileinto", "reject"];

if allof (not allof (address :is ["From","sender"] ["[email protected]","[email protected]"], header :matches "Subject" "INACTIVE*"), address :is "From" "[email protected]")
{
 reject;
}

gets parsed and reconverted to sieve as

require ["fileinto", "reject"];
if allof (not allof (address :is ["From", "sender"] ["[email protected]", "[email protected]"], header :matches "Subject" "INACTIVE*")) {
    reject;
}

thus losing address :is "From" "[email protected]"