cloudevents/spec

CloudEvents Specification

spec's Issues

Show sample json serialization for each context attribute

I think it would be good to show some sample json serialization for each context attribute so there is no ambiguity around what it would look like. It would also be good to show what a complete "context" object would look like. These actually might not be "samples" - by that I mean it might be good to normatively say something like "When serialized in JSON, the format of the event MUST adhere to the following...". We'll see...

Definition: Event

https://github.com/cloudevents/spec/blob/master/spec.md#event

The specification should either drop "occurrence" or nor restate a partial definition in "event".

Suggestion:

An "event" is a data record expressing an occurrence and its context or a set of occurrences and their shared context. Events are routed from the emitting [re]source to interested parties for the purpose of notifying the about the [re]source occurrence. The routing may be performed based on information contained in the event, but an event will generally not identify a specific routing destination.

Assigned to: @clemensv

Specify if event-type should include a version.

Decide and document if the event-type attribute should include a version. A version is currently included in a separate attribute, event-type-version.

"source" name/explanation

"This describes the software instance that emits the event at runtime (i.e. the producer). It contains sub-properties (listed below)"

The description using a "i.e." seems indicative of either the property name being too ambiguous to stand, or the description needing work.

"Software instance" is also often an incorrect term to describe a resource/source/producer, because a resource/source/producer is often a logical concept that may run multiple instances and even be made up of multiple separate implementations that all yield the same events from the same virtual context.

Assigned to: @clemensv

method context attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

method
Type: String
Description: The (http) method used in the call

causation-id context attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

causation-id
Type: String
Description: event-id of event that triggered this event to be created (if any). If not triggered in response to an event, then not present. Enables traceability through systems.
Examples:
Event A generated with unique event-id. Event handler receives event and generates a new event in response, Event B. Event B copies the event-id of Event A into its own causation-id so that the chain of events can be properly correlated.
Constraints:
Optional

Move to cloudevents github organization

System Architecture diagram / document explaining the context for events

We would like to write up some info so that folks can arrive at this repo and understand what kind of event system we're talking about.

Starting docs:

• presentation -- slide 23 (below) shows the difference between general messaging, and "Eventing" which is what CloudEvents is focused on. Events are statements of fact.
  

• https://github.com/cncf/wg-serverless/tree/master/proposals/cloudevents

• Cloud-Native Event Mapping (CNEM)

• Common Event Use Cases

• OpenWhisk has a nice illustration of the kind of system we're talking about CloudEvents:
  

Assigned to: @ultrasaurus @yaronha

Need a better description for schema-url

The description for schema-url is "A link to the schema that the data attribute adheres to". What is meant by "data attribute"?

roadmap with milestones

Proposal to adjust the roadmap to have numbered milestones to help focus our work.

Based on a brainstorm with @ac360:

https://docs.google.com/document/d/1ZLCWPeNcNyjttvgo2QHY9CMtzDaA0_81vmh5C-r9QB4/edit

Assigned to: @ultrasaurus

Remove "references" section

Before we ship a formal release I think we should remove the "References" section that shows what existing providers do for their events. This is very useful today as we work through the initial design, but once we get agreement on our stuff we no longer need to see what other people are doing since it doesn't align with what we're recommending.

Assigned to: @notque

Specify if event-type should enforce a type format

Decide and document if the event-type attribute should include a type format.

Rename attributes 'resource*' to 'source*' or 'producer*'

Rename attributes 'resource*' to 'source*', or alternatively 'producer*', for clarity.

Assigned to: @lfourie

Brainstorm on how to move to a "normal" LGTM PR review process

Since today we don't have a defined set of "maintainers" to approve PRs we decided to follow a model that more akin to how a standard body works. Which is, sort of, ok for now, but not ideal for a variety of reasons - biggest being that not everyone can make the calls.

I'd like to use this issue to track ideas for how we migrate to a "maintainer" process and, in particular, how we identify the initial set of maintainers. It might be as simple as take nominations and vote - I dunno. But let's use this issue to gather ideas and then we'll work on creating a PR with a proposed solution.

Reference section of spec should show CloudEvents version of examples

The examples under the Reference section of the spec are useful, but it would be enlightening for a more outside observer like me if they had a comparison with what those events might look like when transformed into the CloudEvents spec.

contributors list

It would be really helpful to have a list of contributors, to solve two problems:

1. Associating folks who attend Serverless WG meetings with their github username
2. Who is involved in Eventing and is interested in using the CloudEvents spec in some way

Assigned to: @ultrasaurus

Definition of Event in the terminology section is confusing

Definition of Event introduces terms notification and message without clearly defining their meaning in the context of an event-driven system. There may be some implicit assumptions/understanding in there that be explicitly written.

Assigned to: @schatekar

receipt-queue context attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

receipt-queue
Type: String
Description: An optional return endpoint for completion events

Nested vs top-level properties

Should we have nested properties for some things? eg. resource-type and resource-id under "resource". Or should we keep everything at the top-level?

Spec Conformance Tool - Service Validator

Reliable specifications (protocols) require validation and testing to guarantee interoperability, particularly as a specification evolves (versions over time). We're in need of a tool (maybe two?) to validate event formats of operational services to verify their conformance to the normative statements in the CloudEvents specification. To facilitate validation, a checklist of assertions (as a document separate from the CloudEvents spec) may be needed.

Both the conformance tool and list of assertions (requisite tests) should be versioned alongside the CloudEvents specification.

This "CloudEvents Service Validator" would ideally be an open source framework for checking conformance of any generic service implementing the CloudEvents format as defined in the CloudEvents specification. The tool is designed to be vendor/project agnostic and driven purely based on the CloudEvents specification.

Integrity of validation results need to be guaranteed. Therefore, hosting of such a small service validator may be required.

Path Context Attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

path

Type: String
Description: The destination endpoint address (e.g. URL) or target topic

Nested events?

Events occur on different layers of a system. This might lead to nesting of events. E.g. an API gateway may receive an HTTP POST from a web hook, which already contains event data from a SaaS application. In addition the API gateway may be configured to raise an event to a function, when a POST request to a certain URL is received.
How is this situation handled?

Alternative 1: Nesting

The API gateway wraps the SaaS application event into an API gateway event and sends it to the function.

Alternative 2: Forwarding

The API gateway recognizes the event context in the HTTP header and forwards the application event to the function using the protocol of choice. No API gateway event would be triggered.

Alternative 3: ?

Definition: Context

https://github.com/cloudevents/spec/blob/master/spec.md#context

Suggestion for clarification:

A set of metadata attributes included with the event that describe circumstances of the occurrence that are not specific to the occurrence. Tools and application code use context information to identify the relationship of events to aspects of the system or to other events. Context attributes may describe the event source, the originating system, the emitting process, and other circumstantial information.

Assigned to: @clemensv

content-type context attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

content-type
Type: String
Description: The data encoding scheme (e.g. application/json). The data content type.
Constraints:
Required if the event contains a body.

Definition: Occurrence

https://github.com/cloudevents/spec/blob/master/spec.md#occurrence

The definition of "occurrence" is unnecessarily narrowing the scope of what an occurrence may be, especially by calling out specific technologies/protocols.

Suggestion:

An "occurrence" is the capture of a statement of fact during the operation of a software system. This may occur because of a signal raised by the system or a signal being observed by the system, because of a state change, because of a timer elapsing, or any other noteworthy activity.

Assigned to: @clemensv

Revisit the name "source" in Events

On February 1st 2018, we tentatively changed "resource" to "source". We agreed that "resource" is too confusing, but also that "source" may not be the best name. After clarify the meaning of this term we can reaffirm or change the name.

Assigned to: @inlined

Receipt of events that are not understood by event consumer

I think we need to add some description on:

1. nature of an event: critical - must be processed, non-critical - may be ignored, ..
2. action taken when a consumer does not understand a critical event, etc.

log-level context attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

log-level
Type: String
Description: The level of logging that the publisher may want to log this specific event (e.g. debug)

Assigned to: @vbmade2000

Disambiguation/clarification of source/source-id/source-type

The "source" property is, irrespective of being interpreted as a URI or URL, already an identifier for the source itself. The "source-id" therefore appears superfluous.

However, the present schema does not provide any further context qualification for the event relative to the source, so I am proposing to repurpose the source-id to provide that context qualification. In common messaging APIs and protocols (including the emails stack), the qualification property is called "subject".

Furthermore, with "source" already being a structured identifier (URI) and the event being sufficiently qualified by namespace/event-type and the subject, I question the practical value of the "source-type" property and propose for it to be removed.

Assigned to: @clemensv

Specify if event-type can include a namespace prefix

Decide and document if the event-type attribute can include a namespace as a prefix.

The difference among event source, event source type, event type, event ID is not explicit

Suppose the event comes from a AWS S3, what is the event source, event source type, event type, event Identifier? From current definition, it is not very clear.

Assigned to: @cathyhongzhang

Reference Implementation Proposal - CloudEvents Transform Library

Objectives

• Create a simple reference implementation for CloudEvents.
• Create a tool that incentivizes early adoption of the CloudEvents specification.

Overview

This proposal is for a library that transforms common events (e.g. AWS S3 Object Created, webhooks) into the CloudEvents format.

The library could be used by publishers, middleware and subscribers. In the case of serverless, the library could be used at the beginning of serverless function handlers, so the user only needs to work with the CloudEvents format.

Example:

• An "AWS S3 Object Created" event is published on AWS.
• An AWS Lambda function that is subscribed to that event runs.
• The initial code in the AWS Lambda Function does the transformation.
• The user doesn't have to worry about proprietary AWS event formats.

// Node.js Example

// Load the cloudevents Node.js library
var cloudevents = require('cloudevents')

module.exports = function(event, context, callback) {
    
    // Perform transformation of AWS event format into CloudEvents event format
    event = cloudevents.transform(event)
    
    // Write your business logic around the familiar cloudevents format
    return callback(null, 'success')
}

V.0 Specs

• Language agnostic transformation templates should be a major focus of this effort. These can be re-used by cloudevents libraries written in multiple languages. Here is a quick (non-working) example of a template in JSON to communicate the general idea:

// AWS S3 Object Created > CloudEvents
// Keys are CloudEvents properties and values are provided by the original event format

{
    "event-type": "aws.s3.(${eventName}='ObjectCreated:Put'?'object.created')",
    "event-type-version": "${eventVersion}",
    "data": "${s3}"
}

• Transformation templates should offer basic conditional logic.
• Transformation templates should be loaded selectively by the user, to avoid loading irrelevant ones.

Future

• This can be incorporated into a standard FaaS signatures, so that this transformation is always performed.

Does schema-url apply to the whole event or to the data alone?

Decide and document if the schema-url attribute is applicable to the whole event or to the data alone.

Assigned to: @duglin

Based on definition of the content-type, we may just call it "encoding-format" to be more self explanatory

Assigned to: @cathyhongzhang

Add a "What OpenEventing is not" section

This is a question/suggestion. Should we have a "What OpenEvening is not section" in the spec?

Apologies if I have got the wrong end of the stick, but when we say events, a lot of things come to my mind - pub/sub, ESB, queueing systems etc. I don't think we are trying to address "event in every situation" problem. Or are we?

Assigned to: @schatekar

Add clarifying text below Context Attributes Backlog

Add text: This is a list prospective event metadata attributes (“context”) for potential inclusion in the future.

HTTP Mapping Strawman

Just to capture a straw-man for HTTP mappings for discussion purposes - not about the mapping per-se (that might be too early at this point) but to illustrate concrete usage of the properties in the core spec.

With the properties that are in the spec as of today, I'm going to show two candidate mappings for HTTP. One carries an opaque binary payload as-is with the data field being mapped to the HTTP payload, the other carries an event consisting of JSON data. I believe that both scenarios have legitimate use cases.

Upside of the binary mapping is that binary payloads can be routed w/o size penalty. Downside is that you'll have to juggle HTTP headers, which is painful in many environments. Upside up the JSON format is that you have it all in one object, but non-JSON data effectively needs to be treated as binary and base64 encoded.

JSON mapping with JSON data has priority for Microsoft.

Binary: Cloud events properties are mapped to HTTP headers. Content-Type is mapped to the native header.

POST {external} HTTP/1.1
CloudEvents-Namespace: azure.microsoft.com/storage
CloudEvents-EventType: Microsoft.Storage.BlobCreated
CloudEvents-EventTypeVersion: 2018-01-20
CloudEvents-Version: 0.2
CloudEvents-Topic: subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/xstoretestaccount
CloudEvents-Subject: /blobServices/default/containers/oc2d2817345i200097container/blobs/oc2d2817345i20002296blob
CloudEvents-EventId: 831e1650-001e-001b-66ab-eeb76e069631
CloudEvents-EventTime: 2017-06-26T18:41:00.9584103Z
**Content-Type**: application/x-my-opaque-binary
Content-Length: 1234

{Data}

---

JSON: Here, all the cloud events properties are mapped inline at the outer scope of a JSON object. The "data" object contains event-specific data, here an Azure blob store event. I think this will be the more common form.

POST {external} HTTP/1.1
Content-Type: application/json
Content-Length: nnnn

{
    "namespace" : "azure.com"
 /oc2d2817345i20002296blob",
    "cloud-events-version": "0.2"
    "event-type": "Microsoft.Storage.BlobCreated",
    "event-type-version": "1",
    "event-time": "2017-06-26T18:41:00.9584103Z",
    "event-id": "831e1650-001e-001b-66ab-eeb76e069631",
   "topic": "subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/xstoretestaccount",
    "subject": "/blobServices/default/containers/oc2d2817345i200097container/blobs
    "schema-url": "https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/specification/eventgrid/data-plane/Microsoft.Storage/stable/2018-01-01/Storage.json",
    "data": { 
      "api": "PutBlockList",
      "clientRequestId": "6d79dbfb-0e37-4fc4-981f-442c9ca65760",
      "requestId": "831e1650-001e-001b-66ab-eeb76e000000",
      "eTag": "0x8D4BCC2E4835CD0",
      "contentType": "application/octet-stream",
      "contentLength": 524288,
      "blobType": "BlockBlob",
      "url": "https://oc2d2817345i60006.blob.core.windows.net/oc2d2817345i200097container/oc2d2817345i20002296blob",
      "sequencer": "00000000000004420000000000028963",
      "storageDiagnostics": {
        "batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
      }
    }
  }
}

Events in the Actor Model world

I'm trying to get what context this initiative is trying to cover.
I run the Proto.Actor project, cross platform actor framework (http://proto.actor)

For us, things like batching to increase throughput and reduce overall message size is important.
e.g. we buffer messages either until the buffer is full or deadline timeout and then compress the payload by replacing message names/namespaces with identifiers unique for the message batch.

We also heavily rely on things like location transparency, we have the concept of PID (process ID, copied from Erlang, ActorRef in Akka).

Are things like this taken into consideration when designing the spec?

At what level does the spec live, is it down to the raw bits and bytes serialization level?
Or is it more of a checklist that applies above this?
e.g. you can be cloudevent compatible while using either JSON or ProtoBuf or other serializers?

Does event-type-version apply to the entire event or only the data payload?

Decide and document if the event-type-version attribute is applicable to the whole event or to the data payload alone. Currently, event-type-version covers the data, while open-events-version covers the context.

Assigned to: @duglin

Clarify namespace

1. Does namespace identify event or event source (producer)?
2. If latter, what is the value namespace adds on top of source-id?
3. Is there a use case where events with same (event-type, event-type-version) can be published by more than one event source, so with difference in at least one of (source-id, source-type)?
4. If namespace identifies event source and there's value added of namespace on top of source-id would it make sense to include namespace together with (source-id, source-type) as property of source object?

Github repo description references OpenEventing

This repo's name has changed and its contents have been modified to use CloudEvents, but the description of the repository seems to still say "OpenEventing." I found this just a tiny bit confusing and would've put up a PR for it, but I think this needs to be changed by a project admin in the Github interface.

I'm attaching a screenshot for future reference:

authentication context attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

authentication
Type:
Description:

correlation-id context attribute

This is a proposed metadata attribute (from Context Attribute Backlog):

correlation-id
Type: String
Description: Correlation ID of event that triggered this event to be created (if any). If not triggered in response to an event, a unique ID. Can be specified by the producer if event not produced in response to an event. Enables traceability through systems.
Examples:
Event A generated with unique correlation-id. Event handler receives event and generates a new event in response, Event B. Event B copies the correlation-id of Event A so that the chain of events can be properly correlated.
Constraints:
Optional

Assigned to: @cathyhongzhang

CloudEvents Website

This was mentioned in the call today. We're currently hosting a simple site on SquareSpace. Do we want to use something like Hugo and just run it using GitHub Pages? The current site is pretty simple, so it wouldn't take much to convert it.

Here are some potential theme options:

• https://themes.gohugo.io/hugo-theme-lean-launch-page/
• https://themes.gohugo.io/material-docs/
• https://themes.gohugo.io/theme/hugo-base-theme/

These basically represent three options that are quite different from each other, but all can allow the website to grow without changing much as more is added or linked to. The first option is the most basic and is very similar to what exists today. The second is a little more advanced and allows us to link to multiple items in GitHub from the main page. The third is the most elaborate with a picture and a simple blog capability.

I think my preference would be for the second option. I'm happy to do any work to transfer what exists today. I just want to know if there's any interest in this.

Document our release process

What are the steps someone needs to take to “cut a release”?
E.g. tags vs branches, which docs are updated with version #’s...
I would suggest that when cut our first formal release, we just document the steps taken by whoever does it so that the next time people can just follow their lead and we're consistent across releases.

content-type should be explicit

We need the content-type attribute to be explicit (not part ofthe extensions), much like the schema is, without it there is no way to know how to de-serialize the data

Assigned to: @yaronha

Where does the producer specify the format/encoding of the data attribute?

Decide and document where the producer specifies the format/encoding of the data attribute in this schema.

design goals for CloudEvents V.1

Roadmap says: Agree on design goals for CloudEvents V.1. for January

I don't think we've actually accomplished this.

@ac360 got together last week and brainstormed what might be an interim goal that would be useful in aligning a core subset of meta data:

• draft specification that WG members agreed could work
• at least 3 sample events that conform to the specification

Assigned to: @ultrasaurus

Based on the description of "namespace", we may just call it "event-producer" or "event-organization" to avoid confusion.

Assigned to: @cathyhongzhang

May need to add event combination and define a way to express event combination

In some use cases, a function might be triggered to run by a combination of events. For example, a function could be triggered when both event-x and event-y happen or when either event-x or event-y happen. We may need to consider this in this spec?

Assigned to: @cathyhongzhang