Giter Site home page Giter Site logo

brokers's Introduction

Release Slack

TriggerMesh Brokers

TriggerMesh supported brokers.

Configuration

Configuration informs about the Triggers that send events to targets. Durations follow ISO 8601 format

triggers:
  trigger1:
    filters:
    - exact:
        type: example.type
    target:
      url: http://localhost:8888
      deliveryOptions:
        retry: 2
        backoffDelay: PT2S
        backoffPolicy: linear
  trigger2:
    target:
      url: http://localhost:9999
      deliveryOptions:
        retry: 5
        backoffDelay: PT5S
        backoffPolicy: constant
        deadLetterURL: http://localhost:9000

Usage

Produce CloudEvents by sending then using an HTTP client.

curl -v  http://localhost:8080/ \
  -H "Ce-Specversion: 1.0" \
  -H "Ce-Type: example.type" \
  -H "Ce-Source: example.source" \
  -H "Ce-Id: 1234-abcd-x" \
  -H "Content-Type: application/json" \
  -d '{"hello":"broker"}'

Redis

Redis Broker needs a Redis backing server to perform pub/sub operations and storage.

The broker uses a single Redis stream named triggermesh by default, that can be customized using redis.stream argument. The Redis user must be configured to use the stream group of commands on the stream key, plus using the client command with id subcomand for probes.

When using a single Redis backend, it is important to use a unique stream per broker to isolate messages.

# In this example the broker will be configured with user triggermesh1
# and stream name triggermeshstream

ACL SETUSER triggermesh1 on >7r!663R +@stream +client|id ~triggermeshstream

Non Authenticated Redis

# Create storage folder
mkdir -p .local/data

# Run Redis alternative
docker run -d -v $PWD/.local/data:/data \
    -e REDIS_ARGS="--appendonly yes --appendfsync always --rdbcompression yes" \
    --name redis-stack-server \
    -p 6379:6379 \
    redis/redis-stack-server:latest

Launch the broker providing parameters for the backing server.

go run ./cmd/redis-broker start \
  --redis.address "0.0.0.0:6379" \
  --broker-config-path ".local/broker-config.yaml"

Authenticated Redis

When using an authenticated Redis instance, user and password can be informed via redis.username and redis.password arguments.

go run ./cmd/redis-broker start \
  --redis.username triggermesh1 \
  --redis.password "7r\!663R" \
  --redis.address "some.redis.server:25101" \
  --broker-config-path .local/broker-config.yaml

TLS Enabled Redis

If the Redis instance is exposed using TLS, it must enabled at the broker config via redis.tls-enabled flag. For self-signed certificates you can inform them with redis.tls-ca-certificate or skip verification (not recommended) with redis.tls-skip-verify.

go run ./cmd/redis-broker start \
  --redis.username triggermesh1 \
  --redis.password "7r\!663R" \
  --redis.tls-enabled  \
  --redis.tls-ca-certificate="-----BEGIN CERTIFICATE-----abc123-----END CERTIFICATE-----" \
  --redis.address "tls.self.signed.redis.server:25102" \
  --broker-config-path .local/broker-config.yaml

When configuring TLS certificates for Redis authentication, make use of redis.tls-certificate and redis.tls-key.

go run ./cmd/redis-broker start \
  --redis.tls-enabled  \
  --redis.tls-certificate='-----BEGIN CERTIFICATE-----
deadbeef..
-----END CERTIFICATE-----' \
  --redis.tls-key='-----BEGIN PRIVATE KEY-----
c0ff33...
-----END PRIVATE KEY-----' \
  --redis.address "tls.redis.server:25102" \
  --broker-config-path .local/broker-config.yaml

Using Environment Variables

Parameters for the broker can be set as environment variables.

BROKER_CONFIG_PATH=.local/broker-config.yaml \
REDIS_ADDRESS=tls.self.signed.redis.server:25102 \
REDIS_USERNAME=triggermesh1 \
REDIS_PASSWORD=7r\!663R \
REDIS_TLS_ENABLED=true \
REDIS_TLS_SKIP_VERIFY=true \
go run ./cmd/redis-broker start

Note: when using a Redis cluster provide a comma separated list of nodes at REDIS_CLUSTER_ADDRESSES instead of the REDIS_ADDRESS parameter.

Memory

go run ./cmd/memory-broker start --memory.buffer-size 100 --memory.produce-timeout 1s --broker-config-path ".local/config.yaml"

Alternatively environment variables could be used.

CONFIG_PATH=.local/config.yaml MEMORY_BUFFER_SIZE=100 MEMORY_PRODUCE_TIMEOUT=1s go run ./cmd/memory-broker start

Container Images

docker build -t my-repo/redis-broker:my-version .
docker push my-repo/redis-broker:my-version

docker build -t my-repo/memory-broker:my-version .
docker push my-repo/memory-broker:my-version

Observability

The observability-config-path flag allows you to customize observability settings.

go run ./cmd/redis-broker start --redis.address "0.0.0.0:6379" \
  --broker-config-path .local/broker-config.yaml \
  --observability-config-path .local/observability-config.yaml

The file contains a zap-logger-config element where a zap configuration should be located. Updating the file will update the logging level.

zap-logger-config: |
  {
    "level": "info",
    "development": false,
    "outputPaths": ["stdout"],
    "errorOutputPaths": ["stderr"],
    "encoding": "json",
    "encoderConfig": {
      "timeKey": "timestamp",
      "levelKey": "severity",
      "nameKey": "logger",
      "callerKey": "caller",
      "messageKey": "message",
      "stacktraceKey": "stacktrace",
      "lineEnding": "",
      "levelEncoder": "",
      "timeEncoder": "iso8601",
      "durationEncoder": "",
      "callerEncoder": ""
    }
  }

Broker Parameters

Prefixes redis. and memory. apply only to their respective broker binaries.

Name Environment Default Information
broker-config-path BROKER_CONFIG_PATH /etc/triggermesh/broker.conf Path to broker configuration file.
observability-config-path OBSERVABILITY_CONFIG_PATH Path to observability configuration file.
port PORT 8080 HTTP Port to listen for CloudEvents.
broker-name BROKER_NAME {hostname} Instance name. When running at Kubernetes should be set to the pod name.
kubernetes-namespace KUBERNETES_NAMESPACE Namespace where the broker is running.
kubernetes-broker-config-secret-name KUBERNETES_BROKER_CONFIG_SECRET_NAME Secret object name that contains the broker configuration.
kubernetes-broker-config-secret-key KUBERNETES_BROKER_CONFIG_SECRET_KEY Secret object key that contains the broker configuration.
kubernetes-observability-configmap-name KUBERNETES_OBSERVABILITY_CONFIGMAP_NAME ConfigMap object name that contains the observability configuration.
kubernetes-status-configmap-name KUBERNETES_STATUS_CONFIGMAP_NAME ConfigMap object name where the broker instance should write its status.
kubernetes-status-configmap-key KUBERNETES_STATUS_CONFIGMAP_KEY status ConfigMap object key where the broker instance should write its status.
status-reporter-resync-check-period STATUS_REPORTER_RESYNC_CHECK_PERIOD PT10S Period for running status checks for pending changes, using ISO8601.
status-reporter-resync-force-period STATUS_REPORTER_RESYNC_FORCE_PERIOD PT1M Period for running status resync cycles that force status writes, using ISO8601.
config-polling-period CONFIG_POLLING_PERIOD PT0S ISO8601 duration for config polling. Disabled if PT0S. Enabling it will disable other configuration methods.
broker-config BROKER_CONFIG JSON representation of broker configuration. Enabling it will disable other configuration methods.
observability-config BROKER_CONFIG JSON representation of observability configuration. Enabling it will disable other configuration methods.
observability-metrics-domain OBSERVABILITY_CONFIG triggermesh.io/eventing Domain to be used for some metrics reporters.
redis.address REDIS_ADDRESS 0.0.0.0:6379 Redis address for standalone instances.
redis.cluster-addresses REDIS_CLUSTER_ADDRESSES Comma separated list of redis addresses for clustered instances.
redis.username REDIS_USERNAME Redis username.
redis.password REDIS_PASSWORD Redis password.
redis.database REDIS_DATABASE 0 Database ordinal at Redis.
redis.tls-enabled REDIS_TLS_ENABLED false TLS enablement for Redis connection.
redis.tls-skip-verify REDIS_TLS_SKIP_VERIFY false TLS skipping certificate verification.
redis.tls-ca-certificate REDIS_TLS_CA_CERTIFICATE TLS CA certificate used to connect to Redis.
redis.tls-certificate REDIS_TLS_CERTIFICATE TLS certificate used to authenticate with Redis.
redis.tls-key REDIS_TLS_KEY TLS key used to authenticate with Redis.
redis.tracking-id-enabled REDIS_TRACKING_ID_ENABLED false Adds the Redis ID for the event as triggermeshbackendid CloudEvents attribute.
redis.stream REDIS_STREAM triggermesh Stream name that stores the broker's CloudEvents.
redis.group REDIS_GROUP default Redis stream consumer group name.
redis.stream-max-len REDIS_STREAM_MAX_LEN 1000 Limit the number of items in a stream by trimming it. Set to 0 for unlimited.
memory.buffer-size MEMORY_BUFFER_SIZE 10000 Number of events that can be hosted in the backend.
memory.produce-timeout MEMORY_PRODUCE_TIMEOUT PT5S Maximum wait time for producing an event to the backend. Formatted as ISO8601 duration.

Generate License

Install addlicense:

go install github.com/google/[email protected]

Make sure all files contain a license

addlicense -c "TriggerMesh Inc." -y $(date +"%Y") -l apache -s=only ./**/*.go

brokers's People

Contributors

coding-trees avatar dependabot[bot] avatar franbarrera avatar odacremolbap avatar sameersbn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

brokers's Issues

Restructure code.

  • Types might not fit at config package.
  • TestRunner might need its own package.
  • Fakes for each major component will need their own package.

Add license file

Without the root license file, CI validation fails for projects that import this as a module:

E1121 13:24:29.572620   17065 library.go:115] Failed to find license for github.com/triggermesh/brokers/pkg/config/broker: cannot find a known open source license for "/Users/tzununbekov/go/pkg/mod/github.com/triggermesh/[email protected]/pkg/config/broker" whose name matches regexp ^(?i)((UN)?LICEN(S|C)E|COPYING|README|NOTICE).*$ and locates up until "/Users/tzununbekov/go/pkg/mod/github.com/triggermesh/[email protected]"
Error: one or more libraries have an incompatible/unknown license: map["unknown":["github.com/triggermesh/brokers/pkg/config/broker"]]

Add a Docker Compose File

I think the addition of a docker-compose and example broker-config.json would bring a lot here.

Something along the lines of :

services:
  broker:
    image: redisbroker
    build:
      context: .
      dockerfile: cmd/redis-broker/Dockerfile
    command: start
    ports:
      - 8080
    environment:
      BROKER_CONFIG_PATH: /broker-config.yaml 
      REDIS_ADDRESS: redis:6379 
    volumes:
      - ./broker-config-example.yaml:/broker-config.yaml 

  redis:
    container_name: redis
    hostname: redis
    image: redis/redis-stack-server:latest
    environment:
      REDIS_ARGS: --appendonly yes --appendfsync always --rdbcompression yes
    ports:
      - 6379:6379

  redis-commander:
    container_name: redis-commander
    hostname: redis-commander
    image: rediscommander/redis-commander:latest
    build: .
    restart: always
    environment:
      - REDIS_HOSTS=local:redis:6379
    ports:
      - 8089:8081

  sockeye:
    image: docker.io/n3wscott/sockeye:v0.7.0@sha256:e603d8494eeacce966e57f8f508e4c4f6bebc71d095e3f5a0a1abaf42c5f0e48
    ports:
      - 8080:8080
triggers:
  trigger1:
    target:
      url: http://sockeye:8080
      deliveryOptions:
        retry: 2
        backoffDelay: PT2S
        backoffPolicy: linear

Redis: Arbitrary panics when starting the broker

I have seen 3 of these, no some of them informing TLS, some not.
The last relevant change I remember is upgrading go-redis from v9.0.0-rc2 to v9.0.2

2023/03/30 13:34:40 Applying default configuration
{"level":"info","ts":1680176080.2827864,"logger":"fswatch","caller":"fs/filewatcher.go:65","msg":"Adding file to watch","file":"/home/pablo/go/src/github.com/triggermesh/brokers/.local/broker-config.yaml"}
{"level":"info","ts":1680176080.2831702,"logger":"ingest","caller":"ingest/ingest.go:105","msg":"Ingest Server UpdateFromConfig ..."}
{"level":"info","ts":1680176080.2832122,"logger":"subs","caller":"subscriptions/manager.go:52","msg":"Updating subscriptions configuration"}
{"level":"info","ts":1680176080.2832682,"logger":"subs","caller":"subscriptions/manager.go:93","msg":"Creating new subscription from trigger configuration","name":"trigger1","trigger":{"filters":[{"exact":{"type":"example.type"}}],"target":{"url":"http://localhost:9099","deliveryOptions":{"retry":1,"backoffPolicy":"linear","backoffDelay":"PT5S"}}}}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x83f750]

goroutine 1 [running]:
github.com/redis/go-redis/v9/internal/pool.(*ConnPool).waitTurn(0xc0004e6000, {0x0, 0x0})
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/internal/pool/pool.go:284 +0x30
github.com/redis/go-redis/v9/internal/pool.(*ConnPool).Get(0xc0004e6000, {0x0, 0x0})
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/internal/pool/pool.go:245 +0x45
github.com/redis/go-redis/v9.(*baseClient)._getConn(0xc0004641e0, {0x0, 0x0})
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/redis.go:248 +0x38
github.com/redis/go-redis/v9.(*baseClient).getConn(0xc0004641e0, {0x0?, 0x0?})
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/redis.go:236 +0x69
github.com/redis/go-redis/v9.(*baseClient).withConn(0xc0004641e0, {0x0, 0x0}, 0xc0009ae388)
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/redis.go:340 +0x54
github.com/redis/go-redis/v9.(*baseClient)._process(0xc0004641e0, {0x0, 0x0}, {0x1dbb828, 0xc00006ff80}, 0xc000100800?)
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/redis.go:382 +0xe5
github.com/redis/go-redis/v9.(*baseClient).process(0xc0004641e0, {0x0, 0x0}, {0x1dbb828, 0xc00006ff80})
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/redis.go:364 +0x78
github.com/redis/go-redis/v9.(*hooksMixin).processHook(...)
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/redis.go:173
github.com/redis/go-redis/v9.(*Client).Process(0xc0009ae4b0?, {0x0?, 0x0?}, {0x1dbb828?, 0xc00006ff80})
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/redis.go:639 +0x43
github.com/redis/go-redis/v9.cmdable.XGroupCreateMkStream(0xc00052c360, {0x0?, 0x0}, {0xc0004ff100, 0x10}, {0xc000a81300, 0x10}, {0x1b06c79, 0x1})
        /home/pablo/go/pkg/mod/github.com/redis/go-redis/[email protected]/commands.go:1953 +0x1f3
github.com/triggermesh/brokers/pkg/backend/impl/redis.(*redis).Subscribe(0xc00031e0c0, {0xc000a812e0, 0x8}, 0xc0000f2be0)
        /home/pablo/go/src/github.com/triggermesh/brokers/pkg/backend/impl/redis/redis.go:208 +0x1ac
github.com/triggermesh/brokers/pkg/subscriptions.(*Manager).UpdateFromConfig(0xc0008282d0, 0xc00052cc30)
        /home/pablo/go/src/github.com/triggermesh/brokers/pkg/subscriptions/manager.go:99 +0x13b3
github.com/triggermesh/brokers/pkg/config/broker/watcher.(*Watcher).update(0xc000828730, {0xc0000fa600?, 0x4b?, 0xc000542060?})
        /home/pablo/go/src/github.com/triggermesh/brokers/pkg/config/broker/watcher/watcher.go:85 +0x346
github.com/triggermesh/brokers/pkg/config/broker/watcher.(*Watcher).Start(0xc000828730, {0x1db1c70, 0xc000146540})
        /home/pablo/go/src/github.com/triggermesh/brokers/pkg/config/broker/watcher/watcher.go:63 +0xb8
github.com/triggermesh/brokers/pkg/broker.(*Instance).Start(0xc00080c1c0, {0x1db1d18, 0xc0005175c0})
        /home/pablo/go/src/github.com/triggermesh/brokers/pkg/broker/broker.go:255 +0x7a6
github.com/triggermesh/brokers/cmd/redis-broker/cmd.(*StartCmd).Run(0xc0004e09a8, 0xc0004e08c0)
        /home/pablo/go/src/github.com/triggermesh/brokers/cmd/redis-broker/cmd/start.go:33 +0x1d5
reflect.Value.call({0x18c2a80?, 0xc0004e09a8?, 0x18?}, {0x1b077f7, 0x4}, {0xc000338180, 0x1, 0x0?})
        /usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x18c2a80?, 0xc0004e09a8?, 0x18cf1a0?}, {0xc000338180?, 0x1825040?, 0xc0004e0a10?})
        /usr/local/go/src/reflect/value.go:368 +0xbc
github.com/alecthomas/kong.callMethod({0x1b072a3, 0x3}, {0x18f9460?, 0xc0004e09a8?, 0x3?}, {0x18c2a80?, 0xc0004e09a8?, 0x0?}, 0x0?)
        /home/pablo/go/pkg/mod/github.com/alecthomas/[email protected]/callbacks.go:95 +0x549
github.com/alecthomas/kong.(*Context).RunNode(0xc00036b500, 0xc0004f00f0, {0xc0009aff58, 0x1, 0x1})
        /home/pablo/go/pkg/mod/github.com/alecthomas/[email protected]/context.go:755 +0x5f7
github.com/alecthomas/kong.(*Context).Run(0xc0004e08c0?, {0xc0009aff58?, 0xc000793f48?, 0x9?})
        /home/pablo/go/pkg/mod/github.com/alecthomas/[email protected]/context.go:780 +0x14e
main.main()
        /home/pablo/go/src/github.com/triggermesh/brokers/cmd/redis-broker/main.go:48 +0x345
exit status 2

Additional Metadata Needed on Prometheus and Grafana

I am currently exploring the monitoring capabilities of Prometheus/Grafana. While analyzing the timestamp-based events that pass from the source to the broker and then to the target, I have noticed information such as the following:
{__name__="broker_trigger_event_count", broker_name="redis-broker", container="broker", delivered="true", endpoint="metrics", instance="<ip-addresss>:9090", job="monitoring/triggermesh-components", namespace="default", pod="redis-broker-rb-broker-bc55df4b-sm56x", sent_type="com.google.cloud.storage.notification", trigger_name="lambda-trigger"}
To effectively identify the event flow, it would be beneficial to have additional event metadata, such as the event name and other relevant details. This would greatly enhance the identification process and provide better insights into the events being monitored.

Redis: make sure consumer groups are deleted

When a trigger is no longer being used, make sure consumer groups are deleted.

This can be tricky due to multiple instances of a broker operating on the Redis instance. We should at least document if the consumer will keep existing or not.

Add metrics

Primer:

  • Event count
  • Event latency
  • Filter latency

Aggregate on attributes (maybe start with type and source only).

Add concurrency control parameter to Triggers

Use case

There are cases in which you need to control the maximum number of concurrent in-flight events being sent to a target.

For example, 1000 messages land on an SQS queue in one go. The SQS source consumes these messages as fast as it can. The goal is to deliver them to a Service, but the service can only consume at most 10 messages in parallel.

Solution

One idea is to implement a concurrency control on Triggers, such that I can specify the maximum number of non-acknowledged in-flight requests at any given time. Once the limit is reached, the Trigger waits for an event to either be acknowledged by the target or dropped before starting to deliver a new event. This would apply to MemoryBroker and RedisBroker, with different fault tolerance guarantees for each.

The benefit of this solution is that by being on the broker, it can be used regardless of which source connector you're using. But, it requires necessarily using a TriggerMesh broker.

An example of the Trigger configuration could be something like this, which includes the new maxConcurrency parameter that I've tentatively added to the delivery section, alongside retries and dead-letter sink:

apiVersion: eventing.triggermesh.io/v1alpha1
kind: Trigger
metadata:
  name: broker-to-service
spec:
  broker:
    group: eventing.triggermesh.io
    kind: MemoryBroker
    name: mybroker
  target:
    ref:
      apiVersion: v1
      kind: Service
      name: my-service
  delivery:
    maxConcurrency: 10
    deadLetterSink:
      ref:
          apiVersion: v1
          kind: Service
          name: dead-letter-service
    backoffDelay: "PT0.5S"     # ISO8601 duration
    backoffPolicy: exponential # exponential or linear
    retry: 2

An alternative solution would be to implement this on the source connectors themselves, or both, but this is more costly as it requires updates to all source components to reach a consistent feature set.

Handle URI in triggers

Knative triggers support uri for subscription, we should too:

apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  name: my-service-trigger
spec:
  broker: default
  subscriber:
    ref:
      apiVersion: v1
      kind: Service
      name: my-service
    uri: /my-custom-path

[Kafka]Create topic if it doesn't exists

If the topic used by the broker doesn't exists, create it.

Make it optional so that people can pre-create it and do not need to grant topic management permissions to the broker user.

Schema Registry Proposal

Proposal: Include a common package for the brokers that allows them access to user provided schema definitions .

The user can provide the broker with a path to a folder containing CRD's (local or public/internet I.E one could point to https://github.com/triggermesh/triggermesh/tree/48061197f7bacc33477a6576752eeac45978ffee/config) and the package will illiterate over all of the files to look for a registry.knative.dev/eventTypes parameter .

From here the package can make the following information available to the Broker: The event type, schema definition, and a small description.

With the broker now equipped with this information we can begin to leverage it by:

  • Autogenerate swagger endpoint
  • Event type validation.
  • Payload validation.
  • Auto Transformation / AI transformation

[Redis Broker] - Add Certificate Support

Hello,

Thanks for creating the redis broker - I was able to get the sample/demo working against a sample redis instance I spun up and things work great. At my company, we are required to use redis with tls and CA certificate to authenticate and although I do see that the RedisBroker does support tlsEnabled (https://github.com/triggermesh/triggermesh-core/blob/main/config/300-redisbroker.yaml#L90) I noticed that there's no field to put in the CA certificate.

I would like to request an addition to the RedisBroker that would allow to specificy a CA certificate when using RedisBroker - maybe another filed that reads from a secret, similar to the password (https://github.com/triggermesh/triggermesh-core/blob/main/config/300-redisbroker.yaml#L78)?

cacert:
 description: CACert is the Kubernetes secret containing the server CA cert used to connect to the redis instances.
 type: object
 properties:
   secretKeyRef:
      description: A reference to a Kubernetes Secret object.
      type: object
      properties:
         name:
             type: string
         key:
             type: string

Then in the Init (https://github.com/triggermesh/brokers/blob/main/pkg/backend/impl/redis/redis.go#L78) check if it's set and add it if so?

Please forgive me if this has been answered or documented elsewhere, I looked and was not able to find a solution.

Thanks

Restructure filters to add new dialects

Restructure fitlers to add new dialects.
This might mean not importing Knative code but heavily basing on it and exposing at a different level at the API.

Implement poller for config files

Docker at Windows does not work well with fs notify.

We need a poller for broker configuration.
Would be also desirable to have that poller for observability.

Redis Broker - Handle stream deletion of unacknowledged messages

When the Redis stream reaches max length, the oldest messages will be deleted even if they're unacknowledged.
If possible, it would be beneficial to implement a mechanism to handle the deletion of unacknowledged messages by one of the following potential methods:

  • Delete the oldest acknowledged messages, trimming the stream length and allowing for the triggering and acknowledgment of new messages.
  • Stop receiving new messages when the stream only contains unacknowledged messages (should be controlled by a flag to enable the functionality).
  • Trigger the forwarding of the oldest message to the DLQ before deletion when the stream only contains unacknowledged messages (should be controlled by a flag to enable the functionality).

Additionally, it would be beneficial if the Prometheus metrics could include stream metrics (such as XPENDING for example) that would allow setting up alerting based directly on the broker rather than failure handling only once messages are already being dropped.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.