schlagert/syslog

An RFC 3164 and RFC 5424 compliant logging framework for Erlang.

syslog's Issues

Dynamic application names

A useful feature in syslog would be the ability to have more control over the application name when sending messages.

For example if I had the apps foo, bar and baz I'd like to get log messages like this

Jun 13 17:00:00 my_host foo[1111]: A message from the app foo
Jun 13 17:00:00 my_host bar[1111]: A message from the app bar
Jun 13 17:00:00 my_host baz[1111]: A message from the app baz

as opposed to this

Jun 13 17:00:00 my_host foo[1111]: A message from the app foo
Jun 13 17:00:00 my_host foo[1111]: A message from the app bar
Jun 13 17:00:00 my_host foo[1111]: A message from the app baz

The current behavior is to have the name set via either the env var, node name or beam. The specific use case we'd use it for is when we want each app in a release to log with their own name rather than a single name. This makes it easier to track where errors are happening in code, especially since we also forward WARNING messages and above to our monitoring tools.

syslog_logger doesn't handle connected syslog disappearing

While testing #10, I noticed that when I'd shut down the syslog-ng instance I was using that the erlang application would die as well. Here is the stacktrace of it happening

10:14:02.520 [error] gen_server syslog_logger terminated with reason: {error,{tcp_closed,#Port<0.855>}}
10:14:02.521 [error] CRASH REPORT Process syslog_logger with 0 neighbours exited with reason: {error,{tcp_closed,#Port<0.855>}} in gen_server:terminate/7 line 812
10:14:02.521 [error] Supervisor {<0.625.0>,syslog} had child syslog_logger started with syslog_logger:start_link() at <0.626.0> exit with reason {error,{tcp_closed,#Port<0.855>}} in context child_terminated
10:14:02.521 [error] CRASH REPORT Process <0.668.0> with 0 neighbours exited with reason: no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in gen_server:init_it/6 line 352
10:14:02.521 [error] Supervisor {<0.625.0>,syslog} had child syslog_logger started with syslog_logger:start_link() at <0.626.0> exit with reason no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in context start_error
10:14:02.522 [error] CRASH REPORT Process <0.669.0> with 0 neighbours exited with reason: no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in gen_server:init_it/6 line 352
10:14:02.522 [error] Supervisor {<0.625.0>,syslog} had child syslog_logger started with syslog_logger:start_link() at {restarting,<0.626.0>} exit with reason no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in context start_error
10:14:02.523 [error] CRASH REPORT Process <0.670.0> with 0 neighbours exited with reason: no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in gen_server:init_it/6 line 352
10:14:02.523 [error] Supervisor {<0.625.0>,syslog} had child syslog_logger started with syslog_logger:start_link() at {restarting,<0.626.0>} exit with reason no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in context start_error
10:14:02.523 [error] CRASH REPORT Process <0.671.0> with 0 neighbours exited with reason: no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in gen_server:init_it/6 line 352
10:14:02.523 [error] Supervisor {<0.625.0>,syslog} had child syslog_logger started with syslog_logger:start_link() at {restarting,<0.626.0>} exit with reason no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in context start_error
10:14:02.523 [error] CRASH REPORT Process <0.672.0> with 0 neighbours exited with reason: no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in gen_server:init_it/6 line 352
10:14:02.523 [error] Supervisor {<0.625.0>,syslog} had child syslog_logger started with syslog_logger:start_link() at {restarting,<0.626.0>} exit with reason no match of right hand value {error,econnrefused} in syslog_logger:open_device/2 line 272 in context start_error
10:14:02.523 [error] Supervisor {<0.625.0>,syslog} had child syslog_logger started with syslog_logger:start_link() at {restarting,<0.626.0>} exit with reason reached_max_restart_intensity in context shutdown
10:14:02.524 [info] Application syslog exited with reason: shutdown
{"Kernel pid terminated",application_controller,"{application_terminated,syslog,shutdown}"}
Kernel pid terminated (application_controller) ({application_terminated,syslog,shutdown})

Crash dump is being written to: erl_crash.dump...done

It would be good to be able to specify how syslog should handle this disconnection/re-connection situation:

• If it looses the connection then start discarding messages. In my case I'd rather the application continue without logging, than it dying completely
• Have some sort of periodic attempt to re-establish the connection. E.g retry once a minute to see if it can reconnect to the remote syslog server.

Thanks!

{failed_to_start_child,syslog_monitor,noproc} on Erlang 21

Hello. Start of the syslog application fails on Erlang 21. Even the eunit tests fail with Erlang 21 (and succeed with all previous versions). I am getting the following error:

=CRASH REPORT==== 27-Jul-2018::20:29:08.138859 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.305.0>
    registered_name: []
    exception exit: {{shutdown,{failed_to_start_child,syslog_monitor,noproc}},
                     {syslog,start,[normal,[]]}}
      in function  application_master:init/4 (application_master.erl, line 138)
    ancestors: [<0.304.0>]
    message_queue_len: 1
    messages: [{'EXIT',<0.306.0>,normal}]
    links: [<0.304.0>,<0.42.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 376
    stack_size: 27
    reductions: 193
  neighbours:

Could you please have a look on what could be wrong?

STRUCTURED-DATA

Is it possible to utilize structured data fields in this nice syslog

Images for README

Not really an issue. See http://solutionoptimist.com/2013/12/28/awesome-github-tricks/ for details.

Message bottleneck

Hi,

I'm trying to use syslog to talk to Papertrail

https://papertrailapp.com/

Have managed to get the two to talk to one another except that when a large message / group of large messages are sent by my app, communication with Papertrail appears to stop, possibly mid- message; and no further messages are received by Papertrail until the app has been restarted

Setting {no_progress, true} helps somewhat but communication still seems to cease if a large error message / stack trace is sent

It's as if some kind of bottleneck is being reached on the client side; do you have any tips for either increasing the throughput of messages or logging messages to check they are being sent ? I am reasonably sure the issue is on the client side rather than the Papertrail side

Thx,

Justin

syslog_logger_h - failed to process log event

Thanks for this great library. I used this library in my project and sometimes I got following error, do you know how I should resolve it?

syslog_logger_h - failed to process log event #{level => error,meta => 
#{domain => [otp,sasl],error_logger => #{report_cb => fun supervisor:format_log/1,
tag => error_report,type => supervisor_report},file => ....

Adding JSON formatter for the purpose of structured logging.

I want to add custom JSON formatter for the purpuse of structured logging and send the log to the rfc5424 log sever. I've read syslog_logger_h.erl file. But it seems in the following section we didn't add custom formatter and we always use defined formatter.

    {ok, case maps:find(formatter, Cfg2) of
             error ->
                 maps:put(formatter, ?FORMATTER, Cfg2);
             {ok, {logger_formatter, FormatterCfg}}
               when map_size(FormatterCfg) =:= 0 -> %% the OTP default
                 maps:put(formatter, ?FORMATTER, Cfg2);
             {ok, _} ->
                 Cfg2
         end};

incorrect module name for EUnit tests

Thanks for the clean application.

EUnit runs MOD_tests.erl automatically i.e. test module names should end with _tests instead of _test.

IP address forwarded to Papertrail

Hi Tobias,

Your syslog library is working fantastically for me, so many thanks.

However I have one question.

My current config is as follows (obfuscated):

[{syslog, [{dest_host, "logsXXX.papertrailapp.com"},
{dest_port, 12345},
{verbose, false},
{no_progress, true}]}].

Each message received by Papertrail is logged under an 'event', which loosely translates into the hostname / IP address from which the message was received. To quote Papertrail support -

"Papertrail does use the IP when a syslog packet is malformed and we need something to identify it, but otherwise we either use a hardcoded value that's set by the user, or what the sender provides" (for the 'event' name)

Now the weird thing is that it doesn't seem to matter where my app is hosted - whether localhost, EC2 or Linode; Papertrail always logs the event under '127'

Obviously '127' is short for '127.0.0.1'. Now it's possible that there is a problem on the Papertrail side, but the Papertrail guys are saying that I must be sending the messages with 127/127.0.0.1

Is this possible / likely given my config / your understanding of the library ? I have looked through the list of additional config options to see if there's something else I should add, but it's not clear to me what I might change.

Many thanks,

Justin

PS the issue for me is that if everything is logged to '127' by Papertrail, it's difficult/impossible to distinguish between development and production log messages

TIA