Comments (18)
First question: has the program run successfully before and just stopped working recently?
Can you upload the log file for the failed run so I can try to trace what happened.
from dupreport.
Hi there,
re 1.: First run event with DupReport 3.0.4 (Release)
re 2: I am reluctant due to possible privacy issues
I found some time for further investigations:
First run with <= 5 Duplicati Backup Report mails works flawlessly
First run with > 5 Duplicati Backup Report mails yields smtplib.SMTPDataError: (501, b'Syntax error - line too long')
Second run with additional <= 5 Duplicati Backup Report mails works flawlessly
Second run with additional > 5 Duplicati Backup Report mails yields smtplib.SMTPDataError: (501, b'Syntax error - line too long')
from dupreport.
Thanks for the info. Re: the privacy issues, dupReport actually masks private info (email ID, password, etc) in the logs, so it shouldn't reveal anything. You can go into the log (dupReoprt.log) and see for yourself. Nevertheless, let's try to debug without sending anything.
There are two possible causes for this problem:
- One of your emails is too long. Some email servers limit message size.
- A bug in the code we haven't seen before.
We'll start by seeing if #1 is the cause. I suspect that one of your Duplicati runs created an error message, which can sometimes be quite long. Go into your dupReport.rc file and in the [report] section, change the following lines:
[report]
truncatemessage = 100
truncatewarning = 100
truncateerror = 100
truncatelogdata = 100
This will limit any error or warning messages to 100 characters, which should reduce your resulting smtp outgoing email. Run the program and see if that fixes your issue. If not, we'll move on to trying to fix possibility #2.
HG
from dupreport.
Thanks for the new instructions,
re privacy issue: I will look tomorrow for a consent to publish the log file. Thanks for further info towards masking of sensible details.
re #1: I changed the dupReport.rc file in the [report] section according to your instructions, deleted the database and run it again (first run situation). Unfortunately it showed again the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')
from dupreport.
OK, a couple more things to do:
- On the command line, use the option
-f output.html,html
. This will create an output file with the final report called "output.html" that gets written before the email gets sent. If you see the file and it looks normal the problem is somewhere with your SMTP server. - Instead of deleting the database, you can use the
-i
option on the command line. It does the same thing, just saves you a couple of steps and you don't have to manually delete the file every time. - What email/SMTP server are you using? Is it in-house or a commercial system?
from dupreport.
Thanks for these helpful further instructions!
I have now the consent to hand over the log for a first run trial with 7 Duplicati Backup Report Mails in the mail queue.
I issued this command: ./dupReport.py -i -f output.html,html
re 1.: "output.html" is built and looks fine in Browser view although the command execution ended again with the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')
re 3: GMX mail service, a commercial system where it is noted that a mail message including attachment should not exceed >50 MB
from dupreport.
@wechsler42 , can you please rerun the program, but increase the debugging output by adding -v7
to the command line? This will increase the information in the logs to full "DEBUG" level. The program only outputs "NOTICE" level, which for our purposes is not very informative. I apologize for not suggesting -v7
previously.
BTW, the online docs only mention levels 1-3 for the -v option. They are wrong and need to be updated. -v7
will work.
Can you also please upload both the log file and the output.html file. Comparing the two can sometimes help spot errors in the run.
Thanks for your patience.
HG
from dupreport.
Also, it looks like the truncation options mentioned above aren't in the .rc file, as shown in the logs:
[2021-04-09T13:53:28.890302][NOTICE][Report][createReport]Creating report for {'name': 'srcdest', 'type': 'report', 'options': {'layout': [['srcdest'], ['noactivity'], ['lastseen']], 'columns': [['date', 'Date'], ['time', 'Time'], ['dupversion', 'Version'], ['duration', 'Duration'], ['examinedFiles', 'Files'], ['examinedFilesDelta', '+/-'], ['sizeOfExaminedFiles', 'Size'], ['fileSizeDelta', '+/-'], ['addedFiles', 'Added'], ['deletedFiles', 'Deleted'], ['modifiedFiles', 'Modified'], ['parsedResult', 'Result'], ['messages', 'Messages'], ['warnings', 'Warnings'], ['errors', 'Errors'], ['logdata', 'Log Data']], 'title': 'Duplicati Backup Summary Report - By Source/Destination', 'titlebg': '#FFFFFF', 'border': 1, 'padding': 5, 'sizedisplay': 'gb', 'repeatcolumntitles': True, 'suppresscolumntitles': False, 'durationzeroes': True, 'displaymessages': False, 'jobmessagebg': '#FFFFFF', 'displaywarnings': True, 'jobwarningbg': '#FFFF00', 'displayerrors': True, 'joberrorbg': '#FF0000', 'displaylogdata': True, 'truncatemessage': 0, 'truncatewarning': 0, 'truncateerror': 0, 'truncatelogdata': 0, 'joblogdatabg': '#FF0000', 'nobackupwarn': 0, 'nbwsubject': 'Backup Warning: #SOURCE##DELIMITER##DESTINATION# Backup Not Seen for #DAYS# Days', 'groupheadingbg': '#D3D3D3', 'normaldays': 5, 'normalbg': '#FFFFFF', 'warningdays': 20, 'warningbg': '#FFFF00', 'errorbg': '#FF0000', 'weminline': False, 'includeruntime': True, 'failedonly': False, 'showoffline': False, 'type': 'report', 'groupby': [['source', 'ascending'], ['destination', 'ascending']], 'groupheading': 'Source: #SOURCE# - Destination: #DESTINATION#', 'columnsort': [['date', 'ascending'], ['time', 'ascending']]}}.
Please double-check those before the next run to make sure they were changed to something relatively small, like '100'. If you re-check those and they are correct, that's another problem I need to look into. They may be related.
from dupreport.
Hi there,
thanks again for the new debug instructions.
I changed again the dupReport.rc file in the [report] section according to your instructions.
Then I issued the command: ./dupReport.py -i -v7 -f output.html,html
Unfortunately it ended again with the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')
I made the extended log and the output HTML code available. Hope this helps to track down the issue.
from dupreport.
I think I may have found the problem (I hope). I uploaded an "Issue_166" branch to the repository. Please download that code and run it to see if that works. Let me know what happens.
HG
from dupreport.
Hi there,
thanks for your investigations and for providing the "Issue_166" branch I just used for a test run.
I issued the command: ./dupReport.py -i -v7 -f output.html,html
Unfortunately it ended again with the same error: smtplib.SMTPDataError: (501, b'Syntax error - line too long')
I made the extended log available.
I made a second test run with the dupReport.rc file altered in the [report] section according to your instructions. But it ended with the same error. The output.html files looked good in either case.
from dupreport.
Hmmm, that's interesting. Do you have a different email server you can try, just to narrow down where the problem may be?
from dupreport.
I have set up an account on GMX and have been able to recreate the problem. This will make it easier to test potential solutions without the delay of going back & forth. I'll let you know when I have been able to find the cause of the problem.
from dupreport.
I think I have found the issue and have uploaded new code to the Issue_166 branch. Please download that code and give it a try. Let me know what happens.
from dupreport.
Hi there,
thank you very much for fixing the issue. With the latest Issue_166 branch it worked flawlessly 👍
from dupreport.
Great to hear! I suggest letting it run for a few days to make sure you don't have any more problems. If it continues to run well, I'll promote the code to the master branch as an official update.
Thanks for helping with the testing.
from dupreport.
Hi there,
I used the latest Issue_166 branch the last 3 days without any problems. So from my point of view it is ok for an official update. Thanks again for fixing the issue in such a friendly and timely manner.
from dupreport.
You're welcome. I have updated the master branch to version 3.0.5. Thanks for helping me work through this.
from dupreport.
Related Issues (20)
- Add OS Name & Version to Log File HOT 1
- dupReport.py installer changes folder name HOT 2
- Error crash when target destination is unavailable HOT 1
- Typo in [report] section HOT 1
- Change description of -verbose command line option HOT 1
- Program crashes when can't connect to SMTP server HOT 1
- Fix Typos in docs/WhatIsDupreport.md HOT 1
- Indicate Warning or Error in subject line of the mail report HOT 16
- Only send on failure HOT 1
- Parsing failures HOT 4
- Enabling "Less secure apps" is required for Gmail access HOT 2
- "Nonetype has no attribute groups" - dupreport crash HOT 7
- Add an Ignore option to the Source-Destination Section of the .RC file HOT 1
- Need more flexibility in defining Subject Line and S-D regex in .rc file (Was: RegEx suddenly does not match anymore) HOT 16
- dupReport is ignoring Backup Warning Threshold HOT 1
- Typo in the docs for the-b option HOT 1
- Syslog output crashes ELK Logstash 8.1.2 HOT 1
- Ignored S/D Pairs still show up in LastSeen Report
- Error in SafeConfigParser HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dupreport.