[newrelic-logging] Custom Fluentd config

[jsubirat]

[newrelic-logging] Custom Fluentd config

[shicholas]

re: this, another logging service, Loggly, offers an export of our logs to S3. This would solve our issue above nicely as our logs can go from VM => New Relic => Long-term storage (Healthcare, and FinTech companies require long-term log storage) in S3 / Azure Blob Storage / MinIO etc.

[longwa]

We are using containerd as a runtime, which uses the CRI-O log format by default. Fluent Bit has a CRIO parser, but there's no way to configure it via the Helm Chart.

The simplest solution would be to allow to provide an existingConfigMap to at least prevent replacing any custom config each time the chart is upgraded.

[longwa]

Following up, at some point at least support for CRI-O logging format was added for anyone else in this situation. Just add to your values.yaml:

  fluentBit:
    criEnabled: true

To switch the default parser from docker to cri which is defined already in the config.

If you are using the nri-bundle, remember it's under the newrelic-logging sub-chart key:

newrelic-logging:    
  fluentBit:
    criEnabled: true

It might still be nice to be able to provide just a custom ConfigMap for included files, but at least this addresses my direct issue and maybe others.

[luisdavim]

I'd also like to be able to set an extra output for long term storage such as S3, allowing to use an existingConfigMap as mentioned by @longwa or something along the lines of #224 should work.

[jsubirat]

Hi folks,

It is currently possible to add extra inputs, filters or outputs, as now the full Fluent Bit configuration is editable via values.yml: https://github.com/newrelic/helm-charts/blob/master/charts/newrelic-logging/values.yaml#L49-L137

So, you could just use the following for your use cases:

fluentBit:
    config:
        filters: |
      [FILTER]
          Name                kubernetes
          Match               kube.*
          # We need the full DNS suffix as Windows only supports resolving names with this suffix
          # See: https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#dns-limitations
          Kube_URL            https://kubernetes.default.svc.cluster.local:443
          Buffer_Size         ${K8S_BUFFER_SIZE}
          K8S-Logging.Exclude ${K8S_LOGGING_EXCLUDE}
      [FILTER]
          Name           record_modifier
          Match          *
          Record         cluster_name ${CLUSTER_NAME}
      [FILTER]
          [... your extra filter definition here...]

        outputs: |
      [OUTPUT]
          Name           newrelic
          Match          *
          licenseKey     ${LICENSE_KEY}
          endpoint       ${ENDPOINT}
          lowDataMode    ${LOW_DATA_MODE}
          Retry_Limit    ${RETRY_LIMIT}
      [OUTPUT]
          [...your extra OUTPUT definition here...]

Bear in mind that you currently need to include the existing inputs/filters/outputs as well as you own. When we merge this other PR #609, we'll be able to specify just the extra ones instead of rewriting them all.