Comments (8)
Hi @s8sg.
Oh, I found the Register() is very similar to the Update()
I just implemented, and I can revert that change to minimize PR.
However, the root cause I implement the update logic is that I found that there will be panic in this use case:
- At time
$t_1$ , UserA called Exec() to execute FlowA, the FlowA will end at$t_3$ - At time
$t_2$ ($t_1 < t_2 < t_3 $ ), UserB called Register(FlowA, AnotherHandler)
The server process will have panic.
from goflow.
This should be already supported in GoFlow.
Register()
should register the method to define the workflow. The method to define the dag executes when a request hits the go flow executor, but not at the start of the flow (at least this is what expected). This way you can dynamically change the definition of dag by changing/adding additional logic in your define workflow function
from goflow.
Yes. for any active request it will fail.
I think it's unavoidable. It should be taken care by the user to make sure there is no active requests
from goflow.
Even if we keep a snapshot of the dag definition per request and use the snapshot to execute the flow
Say a dag is D with node N1, N2, N3
Say for request R1 that comes at t1 we make D1 as the snapshot as if
D1 -> N1, N2, N3
A DAG can be changed to completely remove a node. Lets say that happens at t2
D -> N1, N2
In that case for the D1 snapshot is no longer valid, and when R1 tries to execute N3 at t3 it will not find the definition
(t1<t2<t3 )
from goflow.
Post completion #4, We should be able to do that without any concern, because workload will be completely independent of the DAG definition
Currently it's in progress. I'll suggest to wait until this feature is added
from goflow.
Even if we keep a snapshot of the dag definition per request and use the snapshot to execute the flow
Say a dag is D with node N1, N2, N3
Say for request R1 that comes at t1 we make D1 as the snapshot as if D1 -> N1, N2, N3
A DAG can be changed to completely remove a node. Lets say that happens at t2 D -> N1, N2
In that case for the D1 snapshot is no longer valid, and when R1 tries to execute N3 at t3 it will not find the definition
(t1<t2<t3 )
I think this is something like transaction concept in databases. This error is similar to read skew. I think in some cases, it is not acceptable if any running workflow is terminated accidentally by user's action. Of course, I didn't dive deep into theoretical background of workflow definition in this project.
Maybe it can be discussed further.
from goflow.
@s8sg Hi, is there any update about #4 ? That is so exciting to have a new version.
from goflow.
Still in progress. Didn’t got much time this weekend. Will try to finish it off my the next. Will require some help with the testing
from goflow.
Related Issues (20)
- [Discussion] Workflow rollback support HOT 1
- [Feature] Add support in server to stop, pause, resume and list flow HOT 1
- [Experimental] GoFlow DSL support
- [Code Quality] Abstract the flow Registry
- [Code Quality] Abstract Queue Worker
- Expose DAG validation in workflow layer HOT 1
- [Feature] Allow to register new flow dynamically HOT 12
- [Question] How to run workflow without middleware independency HOT 4
- [Question] How to set special redis db HOT 2
- There is no option to pass the redis client into goflow struct HOT 1
- [enhancement]: flow events HOT 1
- [enhancement]: support route params and adding middlewares
- [enhancement]: flow metrics / prometheus integration HOT 1
- [Question] How to restart a flow from where it stopped
- Get response from latest edge/node HOT 2
- Broken dependency tree when trying to use goflow for the first time HOT 1
- Why do we choose []byte as the type of workload func's param/output, not interface{} HOT 2
- Dashboard - Missing Data in Requests and Request Monitoring
- Release a new version with Redis password support HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from goflow.