camunda/zeebe

Distributed Workflow Engine for Microservices Orchestration

Zeebe - Workflow Engine for Microservices Orchestration

Zeebe provides visibility into and control over business processes that span multiple microservices. It is the engine that powers Camunda Platform 8.

Why Zeebe?

• Define processes visually in BPMN 2.0
• Choose your programming language
• Deploy with Docker and Kubernetes
• Build processes that react to messages from Kafka and other message queues
• Scale horizontally to handle very high throughput
• Fault tolerance (no relational database required)
• Export process data for monitoring and analysis
• Engage with an active community

Learn more at camunda.com

Release Lifecycle

Our release cadence within major releases is a minor release every six months, with an alpha release on each of the five months between minor releases. Releases happen on the second Tuesday of the month, Berlin time (CET).

Minor releases are supported with patches for eighteen months after their release.

Here is a diagram illustrating the lifecycle of minor releases over a 27-month period:

2022                       2023                                2024
Ap Ma Ju Ju Au Se Oc No De Ja Fe Ma Ap Ma Ju Ju Au Se Oc No De Ja Fe Ma Ap Ma Ju
8.0--------------------------------------------------|
                  8.1--------------------------------------------------|
                                    8.2-----------------------------------------
                                                      8.3-----------------------
                                                                        8.4-----
1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

Here is a diagram illustrating the release schedule of the five alpha releases prior to an upcoming minor release over a 7-month period:

2022                                2023
Oct       Nov          Dec          Jan          Feb          Mar          Apr
8.1-----------------------------------------------------------------------------
          8.2-alpha1   8.2-alpha2   8.2-alpha3   8.2-alpha4   8.2-alpha5   8.2--
1         2            3            4            5            6            7

Status

To learn more about what we're currently working on, check the GitHub issues and the latest commits.

Helpful Links

• Releases
• Pre-built Docker images
• Building Docker images for other platforms
• Blog
• Documentation Home
• Issue Tracker
• User Forum
• Slack Channel
• Contribution Guidelines

Recommended Docs Entries for New Users

• What is Camunda Platform 8?
• Getting Started Tutorial
• Technical Concepts
• BPMN Processes
• Installation and Configuration
• Java Client
• Go Client
• Spring Integration

Contributing

Read the Contributions Guide.

Code of Conduct

This project adheres to the Camunda Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior as soon as possible.

License

Zeebe source files are made available under the Zeebe Community License Version 1.1 except for the parts listed below, which are made available under the Apache License, Version 2.0. See individual source files for details.

Available under the Apache License, Version 2.0:

• Java Client (clients/java)
• Go Client (clients/go)
• Exporter API (exporter-api)
• Protocol (protocol)
• Gateway Protocol Implementation (gateway-protocol-impl)
• BPMN Model API (bpmn-model)

Clarification on gRPC Code Generation

The Zeebe Gateway Protocol (API) as published in the gateway-protocol is licensed under the Zeebe Community License 1.1. Using gRPC tooling to generate stubs for the protocol does not constitute creating a derivative work under the Zeebe Community License 1.1 and no licensing restrictions are imposed on the resulting stub code by the Zeebe Community License 1.1.

zeebe's Issues

Cannot recover ID generator state for workflow instance event log

Implement this properly: https://github.com/camunda-tngp/camunda-tngp/blob/master/broker-core/src/main/java/org/camunda/tngp/broker/wf/runtime/WfInstanceIdReader.java

Core problems:

• events on log may have different types/formats
• per event, there may be more than one id-holding property, out of which the largest value must be selected

related to camunda-tngp/tasks#2

Hack Broker Clustering Component Prototype

No updates for dirty keys

When an update for a particular key is scheduled via the log work buffer, no further updates can be accepted for that key until this update either succeeded or failed.

Notes:

• The update "succeeded" means that a) the log entry has been written and b) the corresponding indices have been updated.
• The pipeline needs to "stall" until the update has been processed downstream

Add infrastructure to write docs

It is never too early to start writing documentation.

At this point we should add infra for writing docs such that when we want to write some we know where to add content

I can configure a consumer id

When starting the client, I can provide a consumer id in the config that is used for all relevant requests. Right now, it always defaults to 0 (based on the behavior that we write requests to all-zero buffers).

Parse a BPMN XML into a process graph

Process graph can be used for executing the model.

related to camunda-tngp/tasks#2

Start process instance client command

See camunda-tngp/tasks#2

Refactor index management

Whenever a log is read and decisions are made based on the log entries, only indexes should be used that reflect the state at the current position. This avoids the need to manage dirty index keys since indexes no longer need to be globally consistent.

Side effects:

• With indexes not reflecting global state, they cannot be used globally. That means, an index that reflects the local state of a log processor can not be used by another log processor or other reader.
• Specifically, request handlers cannot use indexes any longer. Instead, request handling involves only validating a request syntactically and writing a request entry to the addressed log. The log consumer of that log is then going to pick up the request up and decides whether the request is valid or not. Only then can a response be sent.
• Log Consumers can only keep indexes on the log the consume (=> an index on another log would require global consistency). This means, that some events need to be duplicated, e.g. workflow definition creation/deletion must be reflected on workflow instance logs.

Implement DeployBpmnResource Request Reader

Also: decouple BrokerRequestHandlers from SBE protocol

Test Bpmn Process is still deployed after restarting the broker

Workflow continuation task is not restored correctly after broker restart

The BpmnEventHandler holds a LogReader, but that log reader's position is not re-initialized correctly when the broker is restarted. It always begins reading at the log's initial position.

Should be based on the last event that has been handled. Might require a "previous event" pointer for events on the log.

I can set the task queue name for a service task

Currently requires user to specify the task queue id. Should be task queue name, i.e. on broker-side we should resolve the name to an id.

Subtask of camunda-tngp/tasks#4

Delete temp files on shutdown

Currently the broker does not yet delete the log temp files on shut down. It should be a good citizen and clan up after itself.

I can read a getting started guide

Task Type Position Index writer assumes wrong entry length

See: org.camunda.tngp.broker.taskqueue.TaskQueueIndexWriter.updateIndex(long)

Create a pollable task on service task execution

• BPMN specifies task topic
• BPMN specifies task queue
• A task instance is created when an activity instance create event is detected on any workflow instance log. This is the responsibility of the task queue worker (=> single writer to the task instance log).

Subtask of camunda-tngp/tasks#4

Fix inconsistencies due to non-up-to-date indices

Aspects:

• Pipeline Concurrency
• Restore Index on Startup
• Consistency of Log and Indices: Whenever we poll a log, the indices for that log should be up to date (or the poller should be aware of that inconsistency (peek instead of poll))

Returned locked task should include process instance id

That way, clients can managed process context.

Client should throw error when a non-executable process is deployed

Find a safe default port for broker

The default broker port 51015 is sometimes used by applications like Firefox or Outlook (Windows 7). Let's try to find a port with less probability of conflict.

Performance Testsuite

General Questions:

• Why to measure?
• What to measure?
• How to measure?

Other questions:

• What triggers a build?
• When does a build fail?
• How is everything provisioned?

QA: Synchronize client and broker in tests

Client and broker are asynchronous by design.
In test cases, we want a certain synchronization, for example that the client does make a certain request only when the broker has reached a certain state.

I can poll/lock and complete tasks

Clean up and test the code we have right now.

Subtask of #4

LogStreams Api and Implementation

Concepts

Introduce the concept of the "LogStream". A LogStream is a stream of events with the following properties:

• persistent: the stream is persistent. Consumers can go back and re-process events.

The events present in the stream have the following properties:

• unique position within the stream (positions are ascending). Positions are stable and are maintained accross re-org operations such as compaction
• key: groups together a set of events related to the same "thing"
• causedBy: the position of the event which caused this event to occur. This is required for stream processing where the occurrence of an event leads to one or multiple follow-up events being created.

Requirements:

• Pluggable Log Storage
• Concurrency Model: MPMC (multiple producers multiple consumers)
• Batching writes: write operations on the underlying storage are batched to improve performance
• Buffering Reads: read operations read into an intermediary buffer to reduce number of read operations issued to the underlying storage
• Completion: it must be possible to observe the progress of the "log appender" (component responsible for writing to storage.
• Failure notification: when write operations fail, the stream must enter a failure state and ensure consistency by discarding any futher buffered writes until it has recovered from the error.

Test WF Type index writing

We need to tests that the WfType indexes are written correctly. There are two indexes: one on the key and one on the id.

Enforce Coding Style

Test Deploy Bpmn Process Request Handler

• Write tests for the deploy bpmn process resource handler
• Along the way come up with a blueprint of how to test Resquest handlers

In the client, disable payload API

Payload API currently lacks a clear concept, testing and probably functionality. It should be disabled to avoid confusion of users.

Workflow Definition Id vs. Key

AT:

• it is less confusing than in Camunda 7

Remove repository log

• From a user's perspective, this removes the delay between deploying a process model and being able to instantiate it on a runtime log
• When deploying a process model, a target runtime log id must be provided
• Internally, we no longer have to keep redundant log entries (i.e. workflow definition entries on both, repository and runtime logs)

Continue process execution on task completion

• Workflow instance workers monitor task queues for completion events
• trigger activity instance completion for matching task completion events

Subtask of camunda-tngp/tasks#4

Locked task should include absolute lock time

Locktime should then also be included in the Task interface that is handed to TaskHandler implementations (see camunda-tngp/tasks#7).

Workflow Type Log Entry Writer

Come up with a pattern of how to write data to the log

When scanning the log sequentially, staged log entries are considered

Similar to #32

Add constraint validation to BufferWriter implementations

Similar to integrity constraints in relational databases (e.g. check that version of a process definition entry is positive).

I can have non-end-event flow nodes without outgoing sequence flow

Process execution up to a service task

• Create and write events for sequence flow execution and activity instance creation.
• limitation to single outgoing flows

Subtask of camunda-tngp/tasks#4

I can set a workflow resource id when making a client command

Bpmn Validation on Deployment

When a bpmn process is deployed, it is validated to make sure it is executable

I can use a logger in the broker

We need a concept how we want to do logging (i.e. log messages to the console, etc.).

Add Telemetry and Metrics

Start building some telemetry into the system.

Actual requirements need to be determined.

Client should throw error when BPMN XML does not contain exactly one process

Implement start process instance request handler

• There is an endpoint that can receive requests to start a process instance
• This writes two events to the log:
  • start event occurred
  • process instance created
• a response is returned: either success (with process instance id) or failure (with error message and code)

related to camunda-tngp/tasks#2

Parallel runtime and repository requests can fail

We resolve API requests based on log entries. In addition, we rely on the FIFO behavior of requests (i.e. the first deferred request is resolved first). This works when requests target the same log, but not when they target different logs and shared one response pool.

Solution: Process log entries based on the dispatcher (at least API requests). Since all logs use the same dispatcher, this guarantees FIFO behavior for all requests.

Make it easier to write a Perf Test

See comment in #19: it should be possible to provide a "Request" for which I can execute both Latency and Throughput tests.

Understand the Raft Consensus Algorithm

• https://raft.github.io/
• http://thesecretlivesofdata.com/raft/
• https://raft.github.io/raft.pdf

Provide integration test infrastructure

Provide an infrastructure for writing integration tests.

• Broker is started in-process (same VM)
• Java Client is used to write the tests

Provide broker distribution

One of the really good things we have in Camunda 7 is that there is always a downloadable snapshot distribution of the complete platform. We should have that in tngp too.

AT:

• there is a distribution for the tngp broker which can be unzipped and started
• available for Windows and Unix

Clarify and align Naming: Workflow vs. Process

Process execution leaving a service task

Events for activity instance completion, end event occurrence and process completion.

Subtask of camunda-tngp/tasks#4