modoboa/modoboa-stats

Graphical statistics for Modoboa

modoboa-stats

Graphical statistics for Modoboa.

Installation

Install this extension system-wide or inside a virtual environment by running the following command:

$ pip install modoboa-stats

Since this extension relies on the rrdtool package, you may need to install additional libraries in order to compile it. On a Debian system, you will require the following packages:

$ apt-get install librrd-dev

Edit the settings.py file of your modoboa instance and add modoboa_stats inside the MODOBOA_APPS variable like this:

MODOBOA_APPS = (
  'modoboa',
  'modoboa.core',
  'modoboa.lib',
  'modoboa.admin',
  'modoboa.relaydomains',
  'modoboa.limits',
  'modoboa.parameters',
  # Extensions here
  # ...
  'modoboa_stats',
)

Run the following commands to setup the database tables:

$ cd <modoboa_instance_dir>
$ python manage.py collectstatic
$ python manage.py load_initial_data

Finally, restart the python process running modoboa (uwsgi, gunicorn, apache, whatever).

Additional documentation is available on ReadTheDocs.

modoboa-stats's Issues

Restrict stats access to superadmins

Hello,

I recently discoverd that domainAdmins have access to stats, i would prefer they don't :)

Regards,
jC

Statistics don't include alias domains

Mails to alias domains are currently ignored by the log parser. They should be counted and filed in their parent domains bucket.

Wrong langage error message in /stats/

Impacted versions

• Modoboa: 1.13.1

In /stats/ if we type a wrong domain (do not exist) we got an English langage error message (I'm using fr_FR).

Whishlist: have a refresh on /stats/#graphs/?gset=mailtraffic

It will be nice to have an auto refresh on the statistics page.

/stats/#graphs/?gset=mailtraffic

A 60 seconds refresh time will be fine.

Optionally, a global variable to change the refresh time.

Cheers,

Modoboa 1.4.0 and Django 1.8

Ensure compatibility with those versions.

Improve parser performance

From @tonioo on December 4, 2013 15:22

Originally created by Antoine Nguyen on 2009-11-06T11:02:11Z

We need to define a better parser structure:

• Don't parse already handled lines (based on the date)

Copied from original issue: modoboa/modoboa#44

Time entered in 12 hr should be 24 hr

Also reported here: modoboa/modoboa#1545

Impacted versions
Modoboa: 1.10.6
installer used: Yes
Webserver: Nginx
Steps to reproduce
Use the Statistics panel and attempt to set a time between 1PM (13:00) and 12:59 AM (00:59). The AM/PM function and the input field appears to expect 24 hr time format

Current behavior
Unable to set a time between 1PM (13:00) and 12:59 AM (00:59).
Unable to manually edit Date time string, text box not editable.

Expected behavior
Be able to set any time you want.

Video/Screenshot link (optional)
12:07 AM should enter 00:07:SS (SS for seconds it chooses, as seconds cannot be defined in the gui)

3:09 PM has entered 3:09 AM into the text area

Furthest left on graph is 12:30 PM not 12:07 AM as requested

Furthest right on graph is 2:32 AM not 3:09 PM as requested

Stats graph not showing received mail

After a lot of messing about, I've finally gotten everything set up just the way I need it - Thanks for a great project; Modoboa seems to be the best oss mail server front end around!

Strangely, however, my mail traffic graph doesn't show received e-mail, only outgoing e-mail. I've read in a few places that I need to run "manage.py logparser" in a cron job (no cron jobs were set up by the installer), but I have tried running this:

sudo -u modoboa /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser

but the received mail stats still do not show up on the graph. I noticed there was also an "update_statistics" option, but if I try to run that I get this:

Traceback (most recent call last):
File "/srv/modoboa/instance/manage.py", line 22, in
execute_from_command_line(sys.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 364, in execute_from_command_line
utility.execute()
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/update_statistics.py", line 84, in handle
self.update_account_creation_stats(options["rebuild"])
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/update_statistics.py", line 79, in update_account_creation_stats
rrdtool.update(str(db_path), *data)
rrdtool.OperationalError: /srv/modoboa/rrdfiles/new_accounts.rrd: illegal attempt to update using time 1527127200 when last update time is 1527127200 (minimum one second step)

I'm not really sure where to look next - I notice others had problems with the graphs themselves, but the graphs are showing fine for me - I'm just not seeing anything except outgoing emails - the "Accounts" graphs also seem to be fine. Some more info that may be pertinent:

• I have rrdtool installed
• I have letsencrypt running and auto-renewing
• I am on Ubuntu 16.04 Server - a fresh install
• I installed using the installer (not a manual install)
• No crontab has been written (but I have included my own line to renew letsencrypt certs) - not sure if this indicates misconfiguration or not
• Mail in and out seems to be functioning fine
• The only other thing I've installed is roundcube, which is working perfectly

I'd appreciate any help or points in the right direction you can offer!

Thanks in advance -- Stephen

parser is ignoring all lines

on my machine the graph still stays empty after some days of running the new installation. i tried to run the command manually with the following part of the logfile extracted to a separate file:

Oct 19 12:16:40 i7-4770 postfix/postscreen[2107]: CONNECT from [192.30.252.193]:33139 to [178.63.100.119]:25
Oct 19 12:16:40 i7-4770 postfix/postscreen[2107]: WHITELISTED [192.30.252.193]:33139
Oct 19 12:16:40 i7-4770 postfix/smtpd[2108]: connect from out-2.smtp.github.com[192.30.252.193]
Oct 19 12:16:40 i7-4770 postfix/smtpd[2108]: Anonymous TLS connection established from out-2.smtp.github.com[192.30.252.193]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Oct 19 12:16:41 i7-4770 postfix/smtpd[2108]: NOQUEUE: client=out-2.smtp.github.com[192.30.252.193]
Oct 19 12:16:41 i7-4770 postfix/smtpd[2157]: E2CF713001F4: client=localhost.localdomain[127.0.0.1], orig_client=out-2.smtp.github.com[192.30.252.193]
Oct 19 12:16:41 i7-4770 postfix/cleanup[2158]: E2CF713001F4: message-id=<nextcloud/server/pull/11611/[email protected]>
Oct 19 12:16:41 i7-4770 opendkim[2901]: E2CF713001F4: no signing table match for '[email protected]'
Oct 19 12:16:41 i7-4770 opendkim[2901]: E2CF713001F4: signature=yuePZb++ domain=github.com selector=pf2014 result="no signature error"
Oct 19 12:16:41 i7-4770 opendkim[2901]: E2CF713001F4: DKIM verification successful
Oct 19 12:16:41 i7-4770 opendkim[2901]: E2CF713001F4: s=pf2014 d=github.com SSL
Oct 19 12:16:42 i7-4770 postfix/qmgr[10855]: E2CF713001F4: from=<[email protected]>, size=6701, nrcpt=1 (queue active)
Oct 19 12:16:42 i7-4770 dovecot: lmtp(2177): Connect from local
Oct 19 12:16:42 i7-4770 amavis[2719]: (02719-06) Passed CLEAN {RelayedInbound}, [192.30.252.193]:33139 [192.30.252.193] <[email protected]> -> <[email protected]>, Message-ID: <nextcloud/server/pull/11611/[email protected]>, mail_id: q3UTlb9mFDUP, Hits: -0.109, size: 6088, queued_as: E2CF713001F4, dkim_sd=pf2014:github.com, 768 ms
Oct 19 12:16:42 i7-4770 postfix/smtpd[2108]: proxy-accept: END-OF-MESSAGE: 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as E2CF713001F4; from=<[email protected]> to=<[email protected]> proto=ESMTP helo=<out-2.smtp.github.com>
Oct 19 12:16:42 i7-4770 postfix/smtpd[2108]: disconnect from out-2.smtp.github.com[192.30.252.193] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Oct 19 12:16:42 i7-4770 dovecot: lmtp([email protected]): qI6WBQqvyVuBCAAAjSLIHA: sieve: msgid=<nextcloud/server/pull/11611/[email protected]>: stored mail into mailbox 'Backup/Github/nextcloud/server'
Oct 19 12:16:42 i7-4770 postfix/lmtp[2176]: E2CF713001F4: to=<[email protected]>, relay=mail.bigmichi1.de[private/dovecot-lmtp], delay=0.37, delays=0.16/0/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 <[email protected]> qI6WBQqvyVuBCAAAjSLIHA Saved)
Oct 19 12:16:42 i7-4770 dovecot: lmtp(2177): Disconnect from local: Successful quit
Oct 19 12:16:42 i7-4770 postfix/qmgr[10855]: E2CF713001F4: removed

result is

/srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser --debug --logfile a
[parser] ignoring 'postfix' log: 'CONNECT from [192.30.252.193]:33139 to [178.63.100.119]:25'
[parser] ignoring 'postfix' log: 'WHITELISTED [192.30.252.193]:33139'
[parser] ignoring 'postfix' log: 'connect from out-2.smtp.github.com[192.30.252.193]'
[parser] ignoring 'postfix' log: 'Anonymous TLS connection established from out-2.smtp.github.com[192.30.252.193]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)'
[parser] ignoring 'postfix' log: 'E2CF713001F4: client=localhost.localdomain[127.0.0.1], orig_client=out-2.smtp.github.com[192.30.252.193]'
[parser] no log handler for "%r": %r
[parser] no log handler for "%r": %r
[parser] no log handler for "%r": %r
[parser] no log handler for "%r": %r
[parser] ignoring 'amavis' log: '(02719-06) Passed CLEAN {RelayedInbound}, [192.30.252.193]:33139 [192.30.252.193] <[email protected]> -> <[email protected]>, Message-ID: <nextcloud/server/pull/11611/[email protected]>, mail_id: q3UTlb9mFDUP, Hits: -0.109, size: 6088, queued_as: E2CF713001F4, dkim_sd=pf2014:github.com, 768 ms'
[parser] ignoring 'postfix' log: 'proxy-accept: END-OF-MESSAGE: 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as E2CF713001F4; from=<[email protected]> to=<[email protected]> proto=ESMTP helo=<out-2.smtp.github.com>'
[parser] ignoring 'postfix' log: 'disconnect from out-2.smtp.github.com[192.30.252.193] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7'
[parser] ignoring 'postfix' log: 'E2CF713001F4: removed'

Blank page in stats and js error in search

Hello,

The stats page is always blank on a server with small (but some) traffic.

I verify the execution of "python manage.py logparser --verbose" without errors.

The rrd files are created in /tmp/modoboa and are accessible for apache server.

Server is CentOs 7 updated, with rrdtool installed

The js console in a browser shows this error when I search for a nonexisting domain.
TypeError: e is undefined jquery.min.js:3:5099

The versions are:
Statistics 1.2.2
Modoboa 1.9.1
Amavis frontend 1.1.3
Webmail 1.2.2

Some idea to look into?

Thanks.

german umlauts in domains breaks logparser

From @egon0 on March 9, 2018 13:14

Impacted versions

• Modoboa: 1.10.3
• installer used: Yes
• Webserver: Nginx

Steps to reproduce

add a domain or a domain-alias with german umlauts

Current behavior

logparser fails with

UnicodeEncodeError: 'ascii' codec can't encode characters in position 2-3: ordinal not in range(128)

Expected behavior

logparser should parse the logs ;)

Copied from original issue: modoboa/modoboa#1431

Statistics about account creation

Keep track of inactive accounts over time.

Remove spam/virus info from statistics graph if not used

Hi!

It will be nice to remove spam and virus from graphs if virus and spam modules are not installed in the Modoboa server.

Cheers,

Average size graphic broken with Python3

The graphic is always empty.

Legend block is printed on the graphics

Is it possible to avoid this problem?

Problem displaying stats

Hello, stats graphics are not displayed. The logs of the mail.log file are present, the RRD files are also generated. Do you have an idea of the problem ?

Here the json content from query, curves is empty :
{"period_name": "day", "end": 1531389418, "start": "-1day", "period": "day", "graphs": {"averagetraffic": {"curves": [], "title": "Trafic moyen (msgs/min)"}, "averagetrafficsize": {"curves": [], "title": "Taille moyenne du trafic normal (octets/min)"}}, "fname": "global", "domain_selector": true}

Internal Error in stats view

From @fionnb on February 18, 2016 20:25

Hi, on a fresh 1.4.3 install I get errors in the stats module.

File "modoboa_stats/graphics.py", line 94, in export
    timestamp = int(row.find('t').text)

AttributeError: 'NoneType' object has no attribute 'text'

i went digging a bit and had it dump the command, which is:

rrdtool xport --start -1week --end 1455825274 DEF:size_recv=/mail/modoboa/stats/global.rrd:size_recv:AVERAGE CDEF:size_recvpm=size_recv,60,* XPORT:size_recvpm:"Empfangsgr\xc3\xb6sse" DEF:size_sent=/mail/modoboa/stats/global.rrd:size_sent:AVERAGE CDEF:size_sentpm=size_sent,60,* XPORT:size_sentpm:"Versendegr\xc3\xb6sse"

Apparently, when using this command, the result will not contain any 't' fields:

<data>
    <row><v>3.9836091071e+05</v><v>6.2045392857e+04</v></row>
  ...

... and the subsequent code fails. I wonder what goes wrong while constructing the command?

Can I do anything to make it work properly?

Using rrdtool 1.5.5 btw.

Copied from original issue: modoboa/modoboa#836

Let extensions define their own statistics

From @tonioo on December 4, 2013 15:45

Originally created by Antoine Nguyen on 2012-10-21T10:12:17Z

It could be useful to let external users (applications, extensions) define new statistics (graphics).

Copied from original issue: modoboa/modoboa#331

Graph suddenly stopped

Modoboa: 1.13.1
Modoboa-stat: 1.4.1
OS: Debian Stretch

The graph on my production serveur suddenly stopped to work.

Do we have a way to know what is happening?

Python3 compatibility

Make this plugin compatible with Python 3.

Ability to set domains for use with Statiscs module only

From @tonioo on December 4, 2013 15:43

Originally created by Mike C on 2012-07-08T23:11:16Z

I was trying to get Statistics working and had some problems.

If one is only using the "default" and "amavis" Databases but not "pfxadmin" we cannot create domains, maybe because they are not needed?

Except for statics, because the script will "see" all domains as "global" while parsing the log.

Its important to have the possibility to add domains for stats only to get arround this problem.

Copied from original issue: modoboa/modoboa#289

logparser fails to create RRD directory

From @cptMikky on March 5, 2018 23:14

Impacted versions

• Modoboa: 1.10.3
• installer used: No
• Webserver: Nginx (irrelevant)

Steps to reproduce

root@server# /path/to/venv/bin/python /path/to/modoboa/instance/manage.py logparser

Current behavior

Traceback (most recent call last):
  File "./manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 588, in handle
    p.process()
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 559, in process
    self.update_rrd(dom, t)
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 257, in update_rrd
    self.lupdates[fname] = self.init_rrd(fname, m)
  File "/srv/app/modoboa/venv/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 215, in init_rrd
    *params)
rrdtool.OperationalError: Cannot create temporary file

RRD storage directory does not exist and logparser probably doesn't even try to create it.

Expected behavior

Parse log and create RRD files. Create the necessary directory structure and fail properly if that isn't possible.

Command works as expected after the directory is created manually.

Copied from original issue: modoboa/modoboa#1424

Upgrading breaks

Upgrading modoboa to 1.6.2 and stats to 1.0.10 breaks modoboa.
Using uwsgi with nginx
python manage.py makemigrations Illegal instruction

I cannot find any further error messages in nginx or uwsgi. Website gives a gateway timeout.
when modoboa-stats is removed from the extensions, everything works as expected.

Execution of management commands fails

Traceback (most recent call last):
  File "/srv/modoboa/instance/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 190, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 40, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 32, in <module>
    from modoboa.core.management.commands import CloseConnectionMixin
ImportError: cannot import name CloseConnectionMixin

export to JSON directly in graphics.py

In file graphics.py you export to XML and after that you convert to JSON.

I see the comment:

"""Export data to XML using rrdtool and convert it to JSON."""

and you issue the code to export:

       cmd = "{} xport --start {} --end {} ".format(
            self.rrdtool_binary, str(start), str(end))
        cmd += " ".join(cmdargs)
        code, output = exec_cmd(smart_bytes(cmd))
        if code:
            return []

after that you convert to JSON

        for row in tree.xpath('/xport/data/row'):
            for vindex, value in enumerate(row.findall('v')):
                if value.text == 'NaN':
                    result[vindex]['data'].append({'x': timestamp, 'y': 0})
                else:
                    result[vindex]['data'].append(
                        {'x': timestamp, 'y': float(value.text)}
                    )
            timestamp += step
        return result`

According to the man page you can export in JSON-format directly by using the option '--json':
rrdtool xport --json

That's saves code and resources.

The command to directly export to JSON would be:

       cmd = "{} xport --json --start {} --end {} ".format(
            self.rrdtool_binary, str(start), str(end))
        cmd += " ".join(cmdargs)
        code, output = exec_cmd(smart_bytes(cmd))
        if code:
            return []

Modoboa 1.6.1 compatibility

Upgrade for compatibility.

Path to rrdtool should be configurable

The path to the rrdtool binary (used in https://github.com/modoboa/modoboa-stats/blob/master/modoboa_stats/graphics.py#L84) should be configurable to easy deployment in non-standard/-mainstream environments.

journald

with systemd i prefer journald instatt of log-file/syslog

Enhancement: Make parsing regexes configurable

Currently, the regex for detecting spam mail hardcoded to SpamAssassins signature. It wouldbe nice to have this adaptable for Rspamd/Rmilter.

invalid date

Hi!

In the Modoboa statistics my graph is empty. When I mouse-over the graph, it gives an "Invalid date".

When I issue an "rrdtool graph" command from cli, rrdtool plots a png file with valid date.

How can I solve this issue?

Cheers,
Rene

rrdtool.OperationalError: new_accounts.rrd: illegal attempt to update using time (minimum one second step)

Impacted versions

• modoboa==1.10.5
• modoboa-stats==1.4.0
• installer used: yes but not for upgrade
• Webserver: Nginx (irrelevant)

Steps to reproduce

root@server# $PYTHON $INSTANCE/manage.py update_statistics
Traceback (most recent call last):
  File "/srv/modoboa/instance/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/update_statistics.py", line 84, in handle
    self.update_account_creation_stats(options["rebuild"])
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/update_statistics.py", line 79, in update_account_creation_stats
    rrdtool.update(str(db_path), *data)
rrdtool.OperationalError: /srv/modoboa/rrdfiles/new_accounts.rrd: illegal attempt to update using time 1528812000 when last update time is 1528812000 (minimum one second step)

Numerical summary

Would like to see a numerical summary (totals) for sent, received, rejections, bounces, spams, virus, within the selected reporting period (day, week, month, year, custom).

Statistic graph stay in French unless we switch to English

Impacted versions

• modoboa-stats version: 1.4.0
• Distribution: Debian
• Codename: Stretch
• Arch: 64 Bits
• Database: MariadDB

Steps to reproduce

Just click on "Statistics" menu.

Problems:

Type of mail stay in French unless we switch to English.

Do not try to add a new DS when rrdtool < 1.6.0

When people enable the greylist option, the logparser script will try to add a new DS on existing files using rrdtool tune but it does not work with versions < 1.6.0.

Statstistics graph show 2 times in page

Impacted versions

• modoboa-stats version: 1.4.0
• Distribution: Debian
• Codename: Stretch
• Arch: 64 Bits
• Database: MariadDB

Steps to reproduce

Just click on "Statistics" menu.

Problems:

The graph are printed two times.
Type of mail stay in French unless we switch to English.

Can't migrate RRD files

It is not possible to update existing RRD files (when one activates greylisting stats) because the script calls a method which does not exist.

Traceback (most recent call last):
  File "/srv/modoboa/instance/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 585, in handle
    p.process()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 556, in process
    self.update_rrd(dom, t)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 279, in update_rrd
    self.add_point_to_rrd(fname, tpl, values, ts=p)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 237, in add_point_to_rrd
    self.tune_rrd_datasources(str(fname), op_match.group(1))
AttributeError: 'LogParser' object has no attribute 'tune_rrd_datasources'

UnicodeEncodeError while parsing logs

Traceback (most recent call last):
  File "/srv/modoboa/instance/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 589, in handle
    p.process()
  File "/srv/modoboa/env/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 555, in process
    self._parse_line(line)
  File "/srv/modoboa/env/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 546, in _parse_line
    "[parser] no log handler for \"{}\": {}".format(prog, log))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 109: ordinal not in range(128)

FutureWarning: Possible nested set at position 29 with Python 3.7

When running on Python 3.7, the logparser cron job generates a FutureWarning:

<path>/modoboa_stats/management/commands/logparser.py:100: FutureWarning: Possible nested set at position 29
  self._regex = {k: re.compile(v) for k, v in self._regex.items()}