Comments (4)
Hi,
Here is the snippet for the second dynamic task in the main workflow:
{
"name": "dyntask",
"taskReferenceName": "task_2",
"inputParameters": {
"taskToExecute": "workflow.input.task2Name"
},
"type": "DYNAMIC",
"dynamicTaskNameParam": "taskToExecute"
}
The taskToExecute comes as workflow input - so when you start the workflow pass the following input if you want the task_2 to be say "task_3":
{
"task2Name": "task_3"
}
Conductor will use the input as the name of the task and schedule task_3.
from conductor.
Thanks for your help - it now works.
I have some other follow-up questions where I would like to ask you for some more help & information.
What is the typical "API call interaction" when running a workflow?
What I have figured out so far is this
- Start a new workflow using /workflow/{name}.
Then conductor puts the first task (task_1) into the task queue with status "SCHEDULED" - Worker 1 microservice instances (responsible for task_1) periodically poll the task queue using /tasks/poll/task_1 to get new work assigned
This polling changes the status of task_1 to "IN PROGRESS" - Once worker 1 microservice instances have finished the work, they update the status of task_1 using /tasks to e.g. "COMPLETED"
After that, conductor puts the next task(s) into the task queue with status "SCHEDULED" which can be polled by worker microservices again.
=> Is this right so far..? Can you possibly share with me a state machine diagram?
Some other questions came up during my tests:
- Polling some tasks:
- what is the meaning of workerid in this polling API call?
- I noticed that after some time (1 to 2 minutes), it is possible to re-poll a task of the task queue (and the poll count increases).
Which parameter is controlling this behaviour? It seems to be a "time-out" simulation to re-assign work to new worker microservices?
- What is the purpose of /tasks/{taskId}/ack?
What happens to a taskid when running this API call? - Within the UI in the tab "Task Details" there is a field "Callback After" - what is the purpose of this field and how is it related to tasks?
Many thanks for your help.
from conductor.
The state machine you described is correct. I will publish the state machine diagram on the documentation soon.
To answer your questions in the order:
- Worker Id is an informational identifier, we use it with hostname to identify the host on which the worker was running. That helps in troubleshooting issues when worker fails a task.
- Once the task is polled, it needs to be acknowledged by the worker - this is to ensure worker has not lost the polled the task due to network issue etc. If the task is not ack'ed - it will be queued again in 60 seconds. (Configurable but currently fixed)
- When the worker updates the status of the task as
IN_PROGRESS
, it can optionally provide a time delay after which the task should be made available. This is useful for long running tasks where the worker should be periodically checking the status of the task completion (by another process for example) at a fixed interval (say 1 hour). Callback after value can be set by worker as part of the Task's status update.
from conductor.
Thanks a lot. I am looking forward to taking a look at the state machine.
from conductor.
Related Issues (20)
- [FEATURE]: Json schema for input/output data
- [FEATURE]: Groups of tasks/worflows
- Inline Task Getting executed multiple times And giving output as null while it is actually non null HOT 1
- how to make dynamic fork serializer?
- [FEATURE]: Idempotent task execution within fork. HOT 4
- [FEATURE]: Is there any plan to introduce reactive to improve the concurrency capability of the conductor product?
- Task does not time out after retrying once
- UI/UX of Home page HOT 2
- Home page Responsiveness Break
- Subworkflow is completed but subworkflow is timed out HOT 1
- docker-compose up server failed to execute workflow because of "XContentType" class is not found HOT 13
- [FEATURE]: Elasticsearch 8 Compatibility in Netflix Conductor HOT 1
- WAIT Task inside DO_WHILE causing infinite task creation which are already completed HOT 4
- Cannot build docker compose - gradle build error with "libnative-platform-file-events" HOT 2
- Workflow not going forward after DO_WHILE execution completion HOT 5
- Conductor workflow stalled after a sub-workflow HOT 3
- [DOC]: No build.gradle changes required for new Mac with an Apple Silicon Chip when Building Conductor From Source
- Kafka listener implementation
- Conductor Java SDK cannot create a HTTP task with asyncComplete = true
- Same task executed multiple time
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from conductor.