Ideas for an architectural change to add new plugins (db)

[josejuanmontiel]

Ideas for an architectural change to add new plugins (db)

[knadh]

I read something like this in some place under the machinery proyect (maybe the unexplore for me now to the v2 experiment https://github.com/RichardKnop/machinery#v2-experiment) but backing to this proyect... i would like to explore two options:

Machinery is nice. We've used it at work and ended up writing a light weight alternative, https://github.com/kalbhor/Tasqueue which specifically addresses some pitfalls in Machinery. However, it may not be ideal to have to write plugins/servers that depend on a distributed queue. Maybe a simple HTTP webhook system will do? POST from sql-jobber to arbitrary-HTTP-driver-webook endpoint to get results.

1. Standard Go plugin is a really poor system and is only meaningful in a closed environment. All "plugins" have to always be recompiled against master any time the master changes. I'd tried this approach in OTPGateway, but abandoned it for a simple HTTP webhook system because of how impractical and annoying this was: knadh/otpgateway@97baef7

2. WASM is interesting. Haven't toyed with Go/WASM yet.

Thinking about it, a simple HTTP webhook system maybe ideal, just like what I ended up with in OTPGateway. Users can write an HTTP server in any language they prefer and needn't know Go.