dbuenzli/logs

Logging infrastructure for OCaml

logs's Issues

Selectively set the log level of sources from the CLI

In my app, I have two sources of logging, Tamino and Papageno. I use logs.cli to setup the logging of my app in a similar fashion as the example. However, in debug mode, Papageno is extremely verbose and I'm much more interested by what Tamino has to say. Could it be possible to extend the API and find some way to set the verbosity of sources from the command-line interface? I can always add code to disable debug output from Papageno, but that's what I was trying to avoid by switching to logs.cli.
As I understand the API, logs.cli should not set the verbosity of sources directly, but rather return the verbosity set by the user, so it might be difficult to extend the current API (Logs_cli.level) or come up with a new function.
Regarding the actual command-line interface, I think it could look like this:

app --verbosity=debug	# set the debug level for each source
app --verbosity=tamino:debug	# set the verbosity of Tamino to debug. Pamino has the default level.
app --tamino-verbosity=debug	# other syntax proposal
app --verbosity=tamino:debug,pamino:info
app --verbosity=tamino:debug --verbosity=pamino:info
app --tamino-verbosity=debug --pamino-verbosity=info

What style looks the "sanest" to you? I think that only the verbosity flag needs the name expansion.

no tag for 0.7.0 release

0.7.0 release in opam packages and no tag on github
http://opam.ocaml.org/packages/logs/logs.0.7.0

Review interface of Logs_browser

Wanted to make it easy to report to a div instead of the console but the dst parameter is not good for that (header formatting already happened).

provide a basic reporter without dependencies

I find it annoying that I have to write a reporter myself if I don't use Fmt (and all the fancy stuff associated). Is it possible to get a basic reporter packed in Logs itself?

Consider decoupling formatting from output

IIUC the reporter currently has to perform these tasks:

1. call the user supplied 'a msgf to get the actual message
2. take into consideration the log level, and tags to perform the final report (message)
3. actually write the message to a destination

I'd like to be able to change the log destination independently from formatting, e.g. to change between a direct output to a channel, or Lwt write, or syslog, etc.

I wouldn't mind if you add a separate field to type reporter to accomplish this, and another continuation argument to kmsg. The first continuation would be for returning from the report formatter and writer function (currently report field), and the 2nd for signaling completion by the report writing function (the new field proposed here).

Edit: removed comment about composing tag formatting, tags already have their own pretty printers

Note: I don't want to restrict the report writer to strings, so perhaps the report writer should invoke the report formatter with its own Format.formatter (possible printing to a buffer), and once that is finished, write the message in its own way.

Question on best approach to structured logging (json output)

I wanted to have fancy colourful text logging output when running my app interactively, but on the other hand nothing is better than json logs for production instance, so that they can be fed into Elastic or some other log processing system.

Also I wanted to embed some machine-readable fragments to each log message, in a form of json blob for simplicify, and have that added to the end of log messages when running interactively, and blend into log json object when running in production mode.

An example of what I'm trying to achieve:

1558955386.283293 [INFO] Application instance #10 has launched        {"instance_num": 5}

{"ts": "1558955386.283293", "log_level": "info", "message": "Application instance #10 has launched", "data": {"instance_num": 5}}

Probably I need 2 reporters, one for human readable output, another for json. Structured metadata of reach message should probably be a tag with Yojson.t inside (does that sound sane?) I started implementing the first reporter, and got stuck at an attempt to add structured something (so far it's just a string tag for simplicity) to the end of the message. I'm following tags.ml test file as an example, and at line 33 kfprintf is curried with user-provided format string. If I add my %s at the end of format string, that won't work. I was unable to wrap it in a way that does what I want. Any pointers are greatly appreciated, there's not much info on format magic in OCaml around.

Also would be great to get a piece of advice on how to approach the json reporter...

Thanks in advance!

Support for log function genericity

It is currently a bit difficult to define functions that are generic on the log function:

# let f log x y = 
    log (fun m -> m "start"); 
    let v = x + y in 
    log (fun m -> m "result: %d" v); 
    v;;
Error: This expression has type 'a but an expression was expected of type int -> 'a
       The type variable 'a occurs inside int -> 'a

It seems the simplest would be to define:

type 'a Logs.func = { log : 'a. 'a Logs.log }

Functions that want to be generic on log function then need to take a 'a Logs.func argument:

# type 'a func = { log : 'a. 'a Logs.log };;
type 'a func = { log : 'a0. 'a0 Logs.log; }
# let f log x y = 
    log.log (fun m -> m "start"); 
    let v = x + y in 
    log.log (fun m -> m "result: %d" v); 
    v;;
val f : 'a func -> int -> int -> int = <fun>
# f { log = Logs.warn } 4 2;;
ocaml: [WARNING] start
ocaml: [WARNING] result: 6
- : int = 6
# f { log = Logs.err } 4 2;;
ocaml: [ERROR] start
ocaml: [ERROR] result: 6
- : int = 6

Is there maybe a simpler solution that I fail to see ? (/cc @yallop @Drup)

Upstream a ppx

Hello! I created a small ppx to cut down on the boilerplate needed by hiding the fact that there a callback. It looks something like this:
[%log info "Hello %s!" "World"]
Do you want me to upstream it here or should I maintain and publish it on the side?

Thanks for this amazing library btw!

Example is too complex and not complete with regard to dependencies

Had this issue: #18

Had to google to come here.

Wouldn't mind a simpler example... 😅 And please, include the dependencies so I know what to install/include in dune. 👍

Update the install layout to have one library per directory.

This would fix the following issue with dune and has been discussed here.

The example doesn’t compile

Hello,

I’ve got this output when compiling:

➜  slackbot git:(master) ✗ make
ocaml setup.ml -build
Finished, 0 targets (0 cached) in 00:00:00.
+ /Users/paul/.opam/system/bin/ocamlfind ocamlopt -g -safe-string -linkpkg -package cmdliner -package fmt -package logs src/main.cmx -o src/main.native
File "_none_", line 1:
Error: No implementations provided for the following modules:
         Fmt_cli referenced from src/main.cmx
         Logs_fmt referenced from src/main.cmx
         Fmt_tty referenced from src/main.cmx
         Logs_cli referenced from src/main.cmx
Command exited with code 2.
Compilation unsuccessful after building 4 targets (0 cached) in 00:00:00.
E: Failure("Command ''/Users/paul/.opam/system/bin/ocamlbuild' src/main.native -tag debug' terminated with error code 10")
make: *** [build] Error 1

Here’s my _oasis and the source: https://gist.github.com/argent-smith/f212557f937d85a858388f17d75ea119

How to fix this?

Log.on_error[_msg] does not not compose with @@ and |>

This is because ~use is labelled. For example Rresult.R.reword_error_msg does compose with the operators because the error rewording function is unlabelled.

A bit annoying I like the label.

Need a way to pass LDFLAG during build

On armv7l I'm only able to build by using ld.bdf, not ld.gold.

How would I pass LDFLAGS='-fuse-ld=ld.bfd' to an opam install of logs? (I believe this is the traditional way of passing a linker request but maybe there's an ocaml way here?)

(Continuation of #39 )

is there an Lwt_log.file equivalent in Logs ?

Based on this issue ocsigen/lwt#468, it seems that Logs should be preferred over Lwt_log, however after reading the docs, it seems for me (maybe I was not able to understand the doc sorry) that Logs is only for console output and that I can not use it for logging in a file like with Lwt_log.file.

Could you confirm this ?

Switch to Dune build system

Dune is now de-facto default building system for the OCaml programs. It greatly simplifies building, unit testing, and packaging (see dune-release tool).

fails to build on armv7l/ChromeOS

Problem occurs when building wayland-proxy-virtwl

[NOTE] Package wayland-proxy-virtwl is currently pinned to file:///usr/local/tmp/crew/wayland_proxy_virtwl.20211118213136.dir (version ~dev).
wayland-proxy-virtwl is now pinned to file:///usr/local/tmp/crew/wayland_proxy_virtwl.20211118214104.dir (version ~dev)
[wayland-proxy-virtwl.~dev] synchronised (file:///usr/local/tmp/crew/wayland_proxy_virtwl.20211118214104.dir)
The following actions will be performed:
  ∗ install logs                 0.7.0 [required by wayland-proxy-virtwl]
  ∗ install wayland              1.0   [required by wayland-proxy-virtwl]
  ∗ install wayland-proxy-virtwl ~dev*
===== ∗ 3 =====

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
⬇ retrieved logs.0.7.0  (cached)
⬇ retrieved wayland.1.0  (cached)
⬇ retrieved wayland-proxy-virtwl.~dev  (file:///usr/local/tmp/crew/wayland_proxy_virtwl.20211118214104.dir)
[ERROR] The compilation of logs.0.7.0 failed at "ocaml pkg/pkg.ml build --pinned false --with-js_of_ocaml false --with-fmt true --with-cmdliner true --with-lwt true --with-base-threads true".

#=== ERROR while compiling logs.0.7.0 =========================================#
# context              2.1.1 | linux/arm32 | ocaml.4.12.0 | https://opam.ocaml.org#65e6196c
# path                 /usr/local/opam/default/.opam-switch/build/logs.0.7.0
# command              /usr/local/opam/default/bin/ocaml pkg/pkg.ml build --pinned false --with-js_of_ocaml false --with-fmt true --with-cmdliner true --with-lwt true --with-base-threads true
# exit-code            1
# env-file             /usr/local/opam/log/logs-89626-2a4d1e.env
# output-file          /usr/local/opam/log/logs-89626-2a4d1e.out
### output ###
# pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
# [...]
#      'src/logs_fmt.cmx' 'src/logs_fmt.cmi' 'src/logs_fmt.mli'
#      'src/logs_cli.a' 'src/logs_cli.cmxs' 'src/logs_cli.cmxa'
#      'src/logs_cli.cma' 'src/logs_cli.cmx' 'src/logs_cli.cmi'
#      'src/logs_cli.mli' 'src/logs_lwt.a' 'src/logs_lwt.cmxs'
#      'src/logs_lwt.cmxa' 'src/logs_lwt.cma' 'src/logs_lwt.cmx'
#      'src/logs_lwt.cmi' 'src/logs_lwt.mli' 'src/logs_top.a'
#      'src/logs_top.cmxs' 'src/logs_top.cmxa' 'src/logs_top.cma'
#      'src/logs_top.cmx' 'src/logs_threaded.a' 'src/logs_threaded.cmxs'
#      'src/logs_threaded.cmxa' 'src/logs_threaded.cma' 'src/logs_threaded.cmx'
#      'src/logs_threaded.cmi' 'src/logs_threaded.mli' 'src/logs_top_init.ml'
#      'src/logs_fmt_top_init.ml' 'test/tool.ml' 'test/tags.ml']: exited with 10



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build logs 0.7.0
└─ 
╶─ No changes have been performed

Print filename and line number before log

Sometimes I waste a lot of time tracking down the file that's creating a log. Is there a way to set up a formatter to include the filename and line number that triggered a log?

Implement support for Message Templates

There was a request for structured logging in #24 which looks like it was implemented in a separate package as syslog.

It would be great if there was support for structured logging in a json format and it looks like Message Templates is akin to a spec that unifies human readable log messages with structured json messages which looks promising.

Logs.level_{of,to}_string doc

Document the fact that this is compatible with what is parsed by Logs_cli rather than say that the representation is unspecified.

Conflict needed with older cmdliner versions

Appeared on a few opam-ci lower bound tests (e.g https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/5516a097c2586e34975c0260e7009bbfb7723df5/variant/opam-2.1,compilers,4.03,odoc.2.0.0,lower-bounds):

The error we saw is:

# + ocamlfind ocamlc -c -bin-annot -safe-string -package cmdliner -I src -I test -o src/logs_cli.cmi src/logs_cli.mli
# File "src/logs_cli.mli", line 15, characters 17-33:
# Error: Unbound type constructor Cmdliner.Arg.env

with the following downgrades (but the Arg.env type has been there for a while as far as I can see in logs_cli.mli)

  - downgrade cmdliner                 1.0.4 to 0.9.6     [uses ocamlbuild, ocamlfind]
  - downgrade logs                     0.7.0 to 0.6.3     [uses ocamlbuild, ocamlfind]

Review the `result` error loggers

they are not what I want.

Do not use Sys.executable_name

but Sys.argv.(0) as a default prefix in Logs_fmt. Sys.executable_name is ocaml rather than the program name for interpreted programs.

logs 0.7.0 download command failed

Hi folks,

I can't seem to install logs.

I have pinned KaSim to 4.0.0. Is that the cause of the problem?

opam install logs
The following actions will be performed:
  ∗ install logs 0.7.0

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫 
[ERROR] The sources of the following couldn't be obtained, aborting:
          - logs.0.7.0: Download command failed

No `dune` file?

dune seems to be the standard now. Probably this project should use it?

Best way to mix Logs.xxx (fun ..) and Printf.printf ?

Hello, and thanks for the nice library ~

For my application, I'd like to output a progress status, eg:

Printf.printf "\r done %d" counter; flush stdout;

But, it seems that Logs 'grabs' stdout and or hides/buffers output (??). Multi-threaded is enabled, though this does not seem to affect behavior.

How to get the two printing methods to play well together?

Support for 4.01

/cc @talex5

Consider supplying pp_level for convenience

When writing your own reporter it would be helpful if there was a reusable pp_level that takes a function that maps a level to message,color pairs and returns a level Fmt.t

Adapt `logs_top_init.ml` to Omod

# Omod.load "logs";;
[INC] /Users/dbuenzli/.opam/4.03.0/lib/logs
[OBJ] /Users/dbuenzli/.opam/4.03.0/lib/logs/logs.cma
[SRC] /Users/dbuenzli/.opam/4.03.0/lib/logs/logs_top_init.ml
[ERROR] load /Users/dbuenzli/.opam/4.03.0/lib/logs/logs_top_init.ml:
File "/Users/dbuenzli/.opam/4.03.0/lib/logs/logs_top_init.ml", line 1:
Error: Reference to undefined global `Logs_fmt'

- : bool = true

Document how to report to multiple outputs

Is reporting to multiple outputs reasonably easy? What about adding outputs at runtime, with a different formatting for each of them? Currently, a reporter is associated to a formatter and Logs.set_reporter replaces the active reporter. I see no easy way of having multiple active reporters or merging reporters together.

Logs_cli's UI is very bad

Currently the verbosity option has an optional argument.

-q, --quiet
       Be quiet. Takes over --verbose.
-v [LEVEL], --verbose[=LEVEL] (default=info) (absent=warning or TOOL_VERBOSE env)
       Be more or less verbose. LEVEL must be one of `quiet', `error',
       `warning', `info' or `debug'.

However given Cmdliner's flexible command line parsing where options arguments are allowed to be glued or not to the option, this interacts badly with positional arguments. For example:

./tool.native -v arg 
tool: option `-v': invalid value `arg', expected one of `quiet', `error',
         `warning', `info' or `debug'

This problem is of course present with any option that has optional arguments. I actually wonder if supporting options with non-glued optional arguments in Cmdliner was a good idea in the first place, it seems to increase cli's ambiguity.

I think we should have a -v whose repetition simply increases verbosity and a --verbosity=LEVEL option that takes over -v options. -q still takes over both -v and --verbosity.

Add a "trace" reporting level

Hi,

It would be nice to have at least one reporting level below "debug" whose purpose would be to dump very fine grain log data. In the Rust equivalent of this library, it is the "trace" level.

`kmsg` is not sufficient

If one wanted to implement, let's say, Lwt_log (with logging function returning threads), kmsg is not sufficient. You need to provide ksmsg. I guess k would be of type string option -> 'a with None on non-message.

Usage with js_of_ocaml

Here's a minimal example:

open Js_of_ocaml
open Js_of_ocaml_lwt

let on_btn_click _ _ =
    Logs_lwt.debug (fun m -> m "Clicked")

let main _ =
    let btn =
        Js.Opt.get
            (Dom_html.document##getElementById (Js.string "btn"))
            (fun () -> assert false)
    in
    Lwt_js_events.(async (fun () -> clicks btn on_btn_click));
    Js._false


let () =
    Logs.set_reporter (Logs_browser.console_reporter ());
    Logs.set_level (Some Logs.Debug);
    Dom_html.window##.onload := Dom_html.handler main

Compiles fine. When I run this in the browser, I get an error:

Uncaught TypeError: Cannot read property '1' of undefined
    at test.ml:18
    at bigarray.ml:0

Seems to be in this line:

Logs.set_reporter (Logs_browser.console_reporter ())

Am I doing something wrong?

Add notification callbacks on src and tag definition creation

For allowing UIs to react.

Add sample code for tags.

As suggested by @edwintorok

Feature request: a convenient method to raise `Failure` with a logged message

A convenient feature in the Google Logging Library is the FATAL severity level. Please consider adding something to this library with a similar functionality, i.e. something that logs a message to all reporters, then raises Failure with the reported message.

Likewise, something with the same basic behavior but raises Invalid_argument instead of Failure may also be a convenience.

Note: I'm not asking for a new severity level in this library, recognizing that it may or may not be a useful distinction. Just a way to combine logging a message and raising an exception that contains the message.

Note: the Google Logging Library also offers CHECK macros, which are conceptually similar to assert with logging. A similar convenience here might also be worth considering.

Default loggers & message tags

Currently default loggers do not print message tags. Maybe they should.

consider moving the format inside 'a msgf

I'd like to be able to write something like:

Logs.debug (fun m -> m "The value is %d" 42);

Instead of:

Logs.debug "The value is %d" (fun m ->m 42);

It seems more natural to keep the format string and arguments together, although I'm not sure how this change would affect the internals of Logs.