EventCatalog v2 - What's new, thoughts and plan

[boyney123]

EventCatalog v2 - What's new, thoughts and plan

[andy-g]

@boyney123 thanks so much for all your work on EventCatalog.

One of the design aspects that complicates things for us in the current version is the fact that if you want to add consumers you must do this in the Event markdown files. Personally I like the idea that an Event is owned by a producing service, but as in most event systems, it doesn't really care who consumes it. The consumer itself needs to be aware of the event though. In light of that, would it be possible to define the events a consumer is subscribed to in the consuming service itself?

We were looking into the idea of allowing each event producing service to define their own EventCatalog service and the events they produce in their git repo, and similarly allow each consuming service to define their service definition in their own repo (which would ideally include references to the events it consumes). Then in our CI/CD we could pick these configurations up as a artefacts and push it to our central EventCatalog.

If for example a consumer is updated to consume another event, we only need to change the consumer configuration in its repo, this will get pushed to the central EventBridge repo, which can then add this service as a new consumer of the event, without needing to update the event configuration itself - which should really be owned by the event producer.

I hope that makes sense.

[kernwig]

Right now EventCatalog is an easy sell to clients and development teams. They see the value, the learning curve is low, and tossing it up in an S3 bucket and CloudFront is simple and generally costs nothing. (Pennies at most.)

I don't want to lose that. There are hosted platform that cost $10/user/month; I don't even try to justify those to clients.

[debae]

Would be great to have support for multiple environments, I guess most people or companies are having for example DEV / TEST / ... / PROD as environments.
Would be great if you could see which versions of events / schema's are active on which environment.

[thim81]

For the REST API, it would be useful to provide CRUD operations for the users, domains, services, and all the aspects of events. For events specifically:

• bump versions
• download schemas
• edit event properties

People could then build their integration/service/pipeline (next to the plugins), in case they have special transformation needs or certain schema registry tool that they want to combine and centralize in EventCatalog.

[donaldpipowitch]

static site generation will still be possible, correct? Container deployment is an optional feature?

[schrepfler]

Backstage plugin/easy way to integrate with IDPs.

[thim81]

A REST API to manage events, services, ... would open-up extra integration options.

[XaaXaaX]

Hey @boyney123
I think in cases, the domains folder scaffolding was useful, when a domain has many services representing too many events, this will be hard with a flat way , but having markdown side by side for each service seems ok

[boyney123]

static site generation will still be possible, correct? Container deployment is an optional feature?

Hey @donaldpipowitch need to think about that, I think with some server support much more could be done to help govern things. How much would this effect you?

Hey @boyney123 I think in cases, the domains folder scaffolding was useful, when a domain has many services representing too many events, this will be hard with a flat way , but having markdown side by side for each service seems ok

Hey @XaaXaaX yeh should be able to do whatever you want to do. So if you want all events in /events thats fine, if you want events in domains like /domains/mydomain/events/* this should fine. The glob system will fetch all types regardless where they are as long as they would be in an events folder. That's my thinking anyway. So quite flexible!

[boyney123]

Would be great to have support for multiple environments, I guess most people or companies are having for example DEV / TEST / ... / PROD as environments. Would be great if you could see which versions of events / schema's are active on which environment.

Yeah need to figure this out, but 100%.

Backstage plugin/easy way to integrate with IDPs.

Yeah be interested to connect with people that want to integrate with Backstage, EC plugin here, not done much with backstage, do people see EC working along side it or backstage replacing EC?

A REST API to manage events, services, ... would open-up extra integration options.

Yes! Any ideas on the kind of things you would want to do? Things that come to my mind are download schemas, upload new versions of schemas etc,

[donaldpipowitch]

Hey @donaldpipowitch need to think about that, I think with some server support much more could be done to help govern things. How much would this effect you?

It would be awesome, if static export would still be possible. Similar reasons as @kernwig said.

Would be great to have support for multiple environments.

We create an event catalog instance per stage. This seems to be a good approach - especially if you would work in a company where new stages can be dynamically created and removed.

A REST API to manage events, services, ... would open-up extra integration options.

If event catalog wants to support cases like this an approach similar to next-auth might make sense. You create your own Next application and then you just mount any page/api routes offered by event catalog that you need/want.

---

I personally would like to see better support of existing features in v2. Like getting mermaid support in domains (#407) or custom React component usage (similar to how it's done in Storybook).