microsoft/fluentbit-containerd-cri-o-json-log

Parsing CRI JSON logs with Fluent Bit - applies to fluentbit, kubernetes, containerd and cri-o

fluentbit-containerd-cri-o-json-log's Issues

fluentd maintained cri-o vs microsoft

What's the difference between fluentds plugin and yours?

ClusterFlow format

Bug Report

Description

As I have not much experience in fluentd, I struggle a bit to translate your docu into a working ClusterFlow config. I figured starting with something like thi

apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
  name: all-logs
  namespace: cattle-logging-system
  filters:
    - parser:
        key_name: log
        parse:
          expression: >-
            ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*)
            (?<log>.*)$
          time_format: '%Y-%m-%dT%H:%M:%S.%L%z'
  globalOutputRefs:
    - cluster-output-splunk

But don't know where to go from there...

Expected Behavior

N/A

Reproduce

N/A

Additional Context

N/A

Logs emitted under `message` instead of `log`

Bug Report

Description

The provided instructions to create a parser for the cri logs results in the creation of a duplicate parser.

Since 1.0, the fluent-bit docker image has included a built-in list of parsers, one of which is the default cri parser.

Attempting to create another parser with the same name as described here results in an error at start-up:

[error] [parser] parser named 'cri' already exists, skip.

This built-in cri parser emits the main contents of the log under the message field, not the log field, which is contradictory to many of the statements provided.

Expected Behavior

The creation of a parser does not generate errors within fluent-bit.
The logs are emitted under the log key, as described.

Reproduce

1. Take the provided config.yaml and configure a pod running the fluent-bit docker image with a version greater than 1.0.
2. On start-up the pod should emit a log: [error] [parser] parser named 'cri' already exists, skip.
3. The logs output by fluent-bit will have the main contents emitted under message instead of `log.

Additional Context

Recommend that the name of the parser specified in this documentation is changed to something like cri-log, as well as the value for Parser under [INPUT].

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request