cron output

[dehydrated-io]

cron output

[Lukas0907]

I'm using chronic as a wrapper to this script (https://joeyh.name/code/moreutils/). It will output to stdout/stderr only, if the exit code is not 0. This way I'll get an email notification from cron only if an error occurs but not if everything worked as expected.

[WhoHasMyUsername]

That's nice! I didn't know these tools existed. Learned something new.

Meanwhile, I get this, and only this when it's renewal time:

Van: [email protected] (Cron Daemon)
Onderwerp: Cron [email protected] /etc/letsencrypt.sh/letsencrypt.sh --cron --quiet
Datum: 19 maart 2016 03:00:47 CET
Aan: [email protected]

(Less than 30 days). Renewing!

• Generating private key...
• Generating signing request...
• Requesting challenge for example.com...
• Responding to challenge for example.com...
• Challenge is valid!
• Requesting certificate...
• Checking certificate...
• Creating fullchain.pem...
  Reload imap server
  Reloading IMAP/POP3 mail server: dovecot.
  Reload de mailserver
  Reloading Postfix configuration...done.
• Done!

[lukas2511]

In 3111126 I to get all error messages to stderr, so in theory you could just pipe stdout to /dev/null or something, and would still get messages on errors.

I'll leave this ticket open since I think --quiet (-q) and --verbose (-v) options would be nice to have.

[germeier]

My unsorted thoughts:

• One way to solve this would be to channel all messages through a function e.g.:
  msg LOGLEVEL [-n] MESSAGE
• I don't like that the script defaults to cron mode as cron and "interactive" mode have different focus
  • cron mode: only show ERROR and perhaps WARNING messages
  • "interactive" mode/call from the command line: should be more verbose to help the user understand the process of the script
    => I suggest to drop the default mode and revert the default to a short help message and run the script with "-c" from cron.
• I would like to see a logging functionality. Especially when running from cron I only want to get emails when a problem occured but from time to time it would be useful the browse the logfiles to verify my setup works as intended.

[fbender]

I personally like to see a message after a successful --cron renewal, as well as a diagnostic message when the script did run but verified the certs are all still valid. So maybe reduce the messages but at least print a summary of the actions taken, e.g.

Renewed the following certificates:
- domain.tld www.domain.tld
- example.tld www.example.tld
The following certificates are still valid for at least %{RENEW_DAYS} days:
- mail.example.tld

[lukas2511]

@germeier:

i was thinking more about _debug, _info, _warn, _error functions, so it's a bit more readable than "msg 3 whatever", those can also handle writing to a logfile or whatever, and maybe should also format output so that every line begins with "$(date) WARN: ", "$(date) INFO: ", etc.

and i changed default command to help in 1ca883d

[dehnhard]

@lukas2511 Thanks for pointing out stderr/stdout. Although I am still in a process to decid which client to use on my server, this shell script looks good to me and the people friendly - so chances that I'll involve myself here are good.

I thought a bit more about the cron output and it occured to me that most scripts I know use the cron mail facility just for script errors and can additionally send customized mail messages or writes to a log. For example the excellent 'apticron', a cron script for debian updates, can be configured that it sends an email if there are pending updates - even with the changelog, if apt-listchanges is installed. Thats perfect for an admin to receive a mail, if and only if there is a new updateable package.

For letsencrypt.sh I imagine that it could have a configurable mail address to send mail with subject "Certification for domain x successfully renewed" or "Certification renewal failed for x" with details in the body. This way I could setup a mail filter to mark successes as read (I don't need to read them, but now they are present as a kind of history) and if there is an unread mail from this script I am alerted to look into it (which is not the case if every run creates a mail and I have to check each for errors). The standard mail for the cron output could then be reserved for actual runtime errors of the script itself.

How does that sound?

[lukas2511]

I don't want to implement mail functionality in this script, but it should be very easy to build a wrapper around this script to send some mails, either by parsing stdout&stderr or by checking for modifications in certs directory.

[fbender]

Yeah this should be within the cron script that itself calls letsencrypt.sh and redirects STDERR & STDOUT to your mail template.

[WhoHasMyUsername]

Hi,

First; thanks for this script. Much cleaner and simpler and cronnable than the le client.

I would also like a -q | --quiet option. I understand the wrapper-approach but I think -q would be simpler. Am I correct in thinking the -q option is still on the table? I suppose the steps would be:

1. Add _msg function, called with $(level) and $(message), where a date is added and where later writing to a logfile can be handled;
2. Add _debug, _info, _warn, _error functions, called with $(message) that call _msg with the appropiate level;
3. Replace (almost) all echo statements with calls to _debug, _info, _warn, or _error;
4. Add handling for --verbose and --quiet. -v would add debug to info, warnings and errors, -q would show only errors.
5. Have beer.

How about this?

[lukas2511]

Yes this will probably be done at some point.

[WhoHasMyUsername]

Can I do this? I think my coding [quality] would at least be a start but I have never used github before.

[lukas2511]

Sure, you can have a go at this, I don't think anybody is working on this right now.

[WhoHasMyUsername]

Hi, I've added the _msg function and loglevel variable, could you have a look sometime to see if this fits in? It works, i.e. using --quiet or -q silences all output if nothing is to be done.

I still need to add the _debug, _info, _warn, and _error functions I suggested above. I ran into issues with printf statements and the echo ... &>2 statements. I didn't touch the latter.

Let me know.

[lukas2511]

This whole thing gets a lot more complicated than i first thought...

The problem is that normally you wouldn't want to see lines like "Processing example.com" in your command output, because it may not be relevant, but it becomes relevant the moment something happens to that certificate.

That may be solved by actually including the certificate / domain names in every single line of output, but that would make this whole thing really ugly...

I'm thinking about ditching the idea about log-levels, because this makes it way to complicated...

[WhoHasMyUsername]

Ok. I see your point.

Why not use exit codes? A wrapper can capture the output, and only when the exit code > 0, output the output so to speak.

Then again, a script to call a script is not really clean or uncomplicated. The whole (my) point of a bash script for Letsencrypt is automation. And from automation follows a fair requirement to only output messages when relevant.

[lukas2511]

There are already exit codes in place for error cases, and "0" should be returned on success.
You can easily set up a cron that only prints messages on errors, or grep for renewal statements or whatever... It's hard to do this in a clean way inside the script itself.

Yea sure it's great for automation if it would have that, but on the other hand it shouldn't be much work to build a wrapper for this job...
Maybe I'll add a wrapper to the repository at some point.

[WhoHasMyUsername]

I meant adding exit code 2 for renewal, 3 for revoke, but on second thought this is rather unusual for bash scripts. I don't think grepping the output is a clean solution, btw.

Meanwhile, I'm extremely happy with the --quiet option in my fork. :-)

[germeier]

Hi Lukas, hi all!

I have another suggestion for the handling of messages, see: germeier@49c82ef

I added logging to a logfile with different levels for logging (LOGLEVEL) and console output (MSGLEVEL). This way we can make our console output quiet but still have more information in the logfile.

For my usecase LOGLEVEL=3 and MSGLEVEL=2 should work nicely.

What do you think?

[WhoHasMyUsername]

Fwiw, I think it's great.

[lukas2511]

@germeier but this doesn't solve the problem: _info "Processing ${domain}" while having _error "ERROR: ${1}" would result in an error message without the name of the certificate

[germeier]

@lukas2511 How about writing messages below a specified log level to a temporary file and change the msglevel retroactively if the msglevel was reached? untested code: germeier@6da372f

[germeier]

MSGLEVELHIGH=
MSGLEVEL=${MSGLEVELHIGH}

These lines obviously need to be switched. :)

[lukas2511]

Yes buffering the output and replaying it when necessary would work, but it seems ugly :-/

I just wanted to say "me too".

Historically, cron scripts output nothing unless there is an error (which then gets emailed to root or a root alias).

My suggestion, is to send normal output to syslog (journal/systemd) and let the administrator handle it via the existing methods of log handling. I believe this method is more appropriate than creating your own log file, which we admins would have to keep track of and "logrotate" every now and then.

Thank you for this great script.

[rossnick]

@WhoHasMyUsername Where did you take the --quiet option to letsencrypt.sh ? I don't see it in the valid parameters. Also, count me in for this feature !

[lukas2511]

@rossnick --quiet doesn't exist (yet). I'm still hoping on solving this problem at some point, but I still don't feel comfortable with any of the suggestions I got so far.

[MEschenbacher]

cronic is really the type of tool you want to use for this kind of issue. You should set up monitoring for certificate expiry anyways and therefore the regular output of letsencrypt.sh can be supressed unless an error occurs. I do not see a real need for a quiet option.

[rossnick]

I understand, but for it to work, a different return code should be used is a certificat is renewed, which is not the case right now. WIthout that I doupt that cronic can't tell...

[WhoHasMyUsername]

chronic is another tool (to maintain), afaik not installed by default.

-q or --quiet is imho not that outlandish an option.

[MEschenbacher]

@rossnick why do you want to be notified when a certificate renewed successfully? One should set up proper monitoring for certificate lifetimes and not really care about letsencrypt.sh output unless it is erroneous.

I do not have a strong opinion on --quiet as long as the default behaviour is not changed.

[rossnick]

Because on some servers, I don't want to restart/graceful apache unconditiionnaly. On other even a graceful restart is costly in term of load and cpu because of the heavy hits.

[MEschenbacher]

I see. That's where the hooks come into play. See deploy_cert hook in which you can execute commands depending on the cert that has just been successfully deployed.

[rossnick]

Oh ! Thanks @maxlin1990 I never tought of that... It'll be trivial to send an email with a hook script. I could then use a return code check to send an email if there's a problem...

[Aikhjarto]

I like the idea of a "_msg" function instead of a plain "echo" as suggested by @WhoHasMyUsername
However, I don't like the idea of discarding a message because it makes problems hard to find.
I would like to suggest to make "_msg" send all output to a syslog facility instead of discarding it.

[rossnick]

@Aikhjarto For my part, I ended up deploying cronic to all my servers, and used a hook script to send me an email whenever a cert is deployed. I now have much less noise from cron output, and it's viable for me that way.

[lukas2511]

I'm closing this issue. I think that using a wrapper is a lot more elegant than doing the logging inside the script itself.

[rpv-tomsk]

Hi Lukas!

What do you think if some examples of cron setup will be added to Dehydrated documentation?

With suggestions to use one of (at least):

1. chronic (@rossnick, could you please provide an setup example?)
2. /usr/bin/dehydrated -c 1>/dev/null (with explanation what error messages will go to STDERR anyway)

?

This can avoid of reporting new issues/PR like #331.