Job Dependency Supported?

[nextdoor]

Job Dependency Supported?

[wenbinf]

You can structure the code of "Job B" and "Job A" into python functions that are agnostic to ndscheduler. "Job dependency" is achieved by simply calling python functions :)

For our production system, we use ndscheduler together with taskworker, and we also opensource taskworker

When ndscheduler schedules a job, it actually publishes a message to Amazon SQS, then one of taskworker processes would pick up the message from SQS and does the actually work.

When "A runs every 5 minutes and B runs immediately after A", then A is a job scheduled by ndscheduler, while B is not. Both A and B are tasks that run on taskworker. We trigger B from the code of A. Here's the sequence:

1. ndscheduler triggers A and publishes a message to SQS, containing the python class of A and the corresponding arguments passed to the python class in that message.
2. taskworker fetches the message of A from SQS and runs the corresponding python class of A.
3. In the python class of A, it publishes another message to SQS, containing the python class of B.
4. taskworker fetches the message of B from SQS and runs the corresponding python class o B.