Comments (1)
Comment history
Mar 16, 2022 by @thake
If this should be documented in AsyncAPI, it may be better to call it
x-deliveryGuarantees
which is an array of DeliveryGuarantees.
A DeliveryGuarantee is one of these: in-order, no-duplicates. This leaves it open to extensions.
Mar 16, 2022 by @thake
@otto-ec/async-group How is this currently handled? Is there a way that producers specify if they are making sure that events are sent to the queue/topc in order? Does this seem to you like a reasonable thing to document in the contract?
Mar 16, 2022 by @maxedenharter0507
For SNS/SQS based APIs: Standard SNS and SQS does not keep the order of events. For that usecase there is FIFO SNS/SQS which is not compatible with standard SNS/SQS. AFAIK there is no bigger producer which uses FIFO SNS/SQS for public/external APIs. Because of that i think no producer is able to keep the order of events.
Mar 16, 2022 by @thake
With kafka this would be possible using the idempotent producer feature: https://blog.clairvoyantsoft.com/unleash-kafka-producers-architecture-and-internal-working-f33cba6c43aa
Mar 17, 2022 by @slinstaedt
What scope of order are we talking about?
Kafka guarantees strict FIFO on a partition and therefore also on message key basis, which is enough to guarantee happens-before relation for important transactional units like aggregates, given the producer is also serializing command processing for these units, which is desirable even without kafka.
Global message ordering is a different beast though and is hard to archive without killing scaling capabilities.
Mar 18, 2022 by @thake
I'm just talking about ordering guarantees on a partition basis, but again this depends on the chosen transport. Kafka only guarantees strict FIFO on partitions if the producer is configured to be idempotent (see https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#producerconfigs_retries).
Right now I'm leaning towards not including a rule for this for the MVP guidelines.
Mar 18, 2022 by @slinstaedt
Sorry, of course strict FIFO only for idempotent configured consumers.
I leaning in the same direction. If domain event processing in-order or with no-duplicates are mandatory, because for the event's domain out-of-order or duplicates processed events make a reasonable difference, providers should provide capabilities that enable consumers to implement something in this regard themselves. A sequence number could help to do so.
Mar 25, 2022 by @thake
We agreed to specify order guarantees in the AsyncAPI spec. The rule needs to explain why order-guarantees are hard.
Best Practice: Should monitor dropped events.
Mar 30, 2022 by @thake
Currently put on hold because I need a sparring partner to discuss the nitty details of this rule even more. As this is not part of the MVP, it is put on pause.
Mar 30, 2022 by @thake
Already produced content can be found in branch https://github.com/otto-ec/ottoapi_guidelines/tree/feat/producer-garantuees
from api-guidelines.
Related Issues (19)
- [new rule] must use `partitionkey` context attribute (CloudEvents) HOT 1
- [new rule] add a rule(set) to clearly specify how data change events should be used
- [new rule] must provide compatibility mode for produced messages
- [update] rework ETag usage
- [new rule] compacted topics must always be ordered
- [define] usage of CloudEvents' `dataschema` conflicts with the Confluent wire format HOT 1
- [update] relax "MUST change type context attribute to indicate incompatible changes" to allow deprecation cycle on property basis HOT 1
- [define] evaluate possibilities for SNS/SQS CloudEvents binding HOT 1
- [define] support for commands HOT 2
- [update] definition of domain event
- [update] add example for MAY provide sequence
- [new rule] write domain events originating from the same entity type to the same kafka topic
- [new rule] guidelines for avro format
- [new rule] create rule for formatting prices HOT 16
- [update] remove rule MUST NOT validate audience of the JSON Web Token HOT 1
- [define] evaluate proposed w3c standard "Content Negotiation by Profile"
- [orga] add issue template
- [update] use link shortener
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from api-guidelines.