smtplib.SMTPDataError: (501, b'Syntax error - line too long')

[wechsler42]

smtplib.SMTPDataError: (501, b'Syntax error - line too long')

[HandyGuySoftware]

First question: has the program run successfully before and just stopped working recently?

Can you upload the log file for the failed run so I can try to trace what happened.

[wechsler42]

Hi there,
re 1.: First run event with DupReport 3.0.4 (Release)
re 2: I am reluctant due to possible privacy issues

I found some time for further investigations:
First run with <= 5 Duplicati Backup Report mails works flawlessly
First run with > 5 Duplicati Backup Report mails yields smtplib.SMTPDataError: (501, b'Syntax error - line too long')
Second run with additional <= 5 Duplicati Backup Report mails works flawlessly
Second run with additional > 5 Duplicati Backup Report mails yields smtplib.SMTPDataError: (501, b'Syntax error - line too long')

[HandyGuySoftware]

Thanks for the info. Re: the privacy issues, dupReport actually masks private info (email ID, password, etc) in the logs, so it shouldn't reveal anything. You can go into the log (dupReoprt.log) and see for yourself. Nevertheless, let's try to debug without sending anything.

There are two possible causes for this problem:

1. One of your emails is too long. Some email servers limit message size.
2. A bug in the code we haven't seen before.

We'll start by seeing if #1 is the cause. I suspect that one of your Duplicati runs created an error message, which can sometimes be quite long. Go into your dupReport.rc file and in the [report] section, change the following lines:

[report]
truncatemessage = 100
truncatewarning = 100
truncateerror = 100
truncatelogdata = 100

This will limit any error or warning messages to 100 characters, which should reduce your resulting smtp outgoing email. Run the program and see if that fixes your issue. If not, we'll move on to trying to fix possibility #2.

HG

[wechsler42]

Thanks for the new instructions,

re privacy issue: I will look tomorrow for a consent to publish the log file. Thanks for further info towards masking of sensible details.

re #1: I changed the dupReport.rc file in the [report] section according to your instructions, deleted the database and run it again (first run situation). Unfortunately it showed again the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')

[HandyGuySoftware]

OK, a couple more things to do:

1. On the command line, use the option -f output.html,html. This will create an output file with the final report called "output.html" that gets written before the email gets sent. If you see the file and it looks normal the problem is somewhere with your SMTP server.
2. Instead of deleting the database, you can use the -i option on the command line. It does the same thing, just saves you a couple of steps and you don't have to manually delete the file every time.
3. What email/SMTP server are you using? Is it in-house or a commercial system?

[wechsler42]

Thanks for these helpful further instructions!
I have now the consent to hand over the log for a first run trial with 7 Duplicati Backup Report Mails in the mail queue.
I issued this command: ./dupReport.py -i -f output.html,html
re 1.: "output.html" is built and looks fine in Browser view although the command execution ended again with the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')
re 3: GMX mail service, a commercial system where it is noted that a mail message including attachment should not exceed >50 MB

[HandyGuySoftware]

@wechsler42 , can you please rerun the program, but increase the debugging output by adding -v7 to the command line? This will increase the information in the logs to full "DEBUG" level. The program only outputs "NOTICE" level, which for our purposes is not very informative. I apologize for not suggesting -v7 previously.

BTW, the online docs only mention levels 1-3 for the -v option. They are wrong and need to be updated. -v7 will work.

Can you also please upload both the log file and the output.html file. Comparing the two can sometimes help spot errors in the run.

Thanks for your patience.

HG

[HandyGuySoftware]

Also, it looks like the truncation options mentioned above aren't in the .rc file, as shown in the logs:

[2021-04-09T13:53:28.890302][NOTICE][Report][createReport]Creating report for {'name': 'srcdest', 'type': 'report', 'options': {'layout': [['srcdest'], ['noactivity'], ['lastseen']], 'columns': [['date', 'Date'], ['time', 'Time'], ['dupversion', 'Version'], ['duration', 'Duration'], ['examinedFiles', 'Files'], ['examinedFilesDelta', '+/-'], ['sizeOfExaminedFiles', 'Size'], ['fileSizeDelta', '+/-'], ['addedFiles', 'Added'], ['deletedFiles', 'Deleted'], ['modifiedFiles', 'Modified'], ['parsedResult', 'Result'], ['messages', 'Messages'], ['warnings', 'Warnings'], ['errors', 'Errors'], ['logdata', 'Log Data']], 'title': 'Duplicati Backup Summary Report - By Source/Destination', 'titlebg': '#FFFFFF', 'border': 1, 'padding': 5, 'sizedisplay': 'gb', 'repeatcolumntitles': True, 'suppresscolumntitles': False, 'durationzeroes': True, 'displaymessages': False, 'jobmessagebg': '#FFFFFF', 'displaywarnings': True, 'jobwarningbg': '#FFFF00', 'displayerrors': True, 'joberrorbg': '#FF0000', 'displaylogdata': True, 'truncatemessage': 0, 'truncatewarning': 0, 'truncateerror': 0, 'truncatelogdata': 0, 'joblogdatabg': '#FF0000', 'nobackupwarn': 0, 'nbwsubject': 'Backup Warning: #SOURCE##DELIMITER##DESTINATION# Backup Not Seen for #DAYS# Days', 'groupheadingbg': '#D3D3D3', 'normaldays': 5, 'normalbg': '#FFFFFF', 'warningdays': 20, 'warningbg': '#FFFF00', 'errorbg': '#FF0000', 'weminline': False, 'includeruntime': True, 'failedonly': False, 'showoffline': False, 'type': 'report', 'groupby': [['source', 'ascending'], ['destination', 'ascending']], 'groupheading': 'Source: #SOURCE# - Destination: #DESTINATION#', 'columnsort': [['date', 'ascending'], ['time', 'ascending']]}}.

Please double-check those before the next run to make sure they were changed to something relatively small, like '100'. If you re-check those and they are correct, that's another problem I need to look into. They may be related.

[wechsler42]

Hi there,
thanks again for the new debug instructions.
I changed again the dupReport.rc file in the [report] section according to your instructions.
Then I issued the command: ./dupReport.py -i -v7 -f output.html,html
Unfortunately it ended again with the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')
I made the extended log and the output HTML code available. Hope this helps to track down the issue.

[HandyGuySoftware]

I think I may have found the problem (I hope). I uploaded an "Issue_166" branch to the repository. Please download that code and run it to see if that works. Let me know what happens.

HG

[wechsler42]

Hi there,
thanks for your investigations and for providing the "Issue_166" branch I just used for a test run.
I issued the command: ./dupReport.py -i -v7 -f output.html,html
Unfortunately it ended again with the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')
I made the extended log available.
I made a second test run with the dupReport.rc file altered in the [report] section according to your instructions. But it ended with the same error. The output.html files looked good in either case.

[HandyGuySoftware]

Hmmm, that's interesting. Do you have a different email server you can try, just to narrow down where the problem may be?

[HandyGuySoftware]

I have set up an account on GMX and have been able to recreate the problem. This will make it easier to test potential solutions without the delay of going back & forth. I'll let you know when I have been able to find the cause of the problem.

[HandyGuySoftware]

I think I have found the issue and have uploaded new code to the Issue_166 branch. Please download that code and give it a try. Let me know what happens.

[wechsler42]

Hi there,
thank you very much for fixing the issue. With the latest Issue_166 branch it worked flawlessly 👍

[HandyGuySoftware]

Great to hear! I suggest letting it run for a few days to make sure you don't have any more problems. If it continues to run well, I'll promote the code to the master branch as an official update.

Thanks for helping with the testing.

[wechsler42]

Hi there,
I used the latest Issue_166 branch the last 3 days without any problems. So from my point of view it is ok for an official update. Thanks again for fixing the issue in such a friendly and timely manner.

[HandyGuySoftware]

You're welcome. I have updated the master branch to version 3.0.5. Thanks for helping me work through this.