ScHeduling Algorithms for Data-intensive Workflows (shadow)
shadow is a library for the use and testing of workflow scheduling algorithms, with a focus on data-intensive science applications. shadow provides implementations of various heuristic and metaheuristic algorithms to address single-and multi-objective scheduling problems; these algorithms are accessed using a workflow-oriented interface the library. shadow also comes with a command-line interface that allows you to run these algorithms on pipelines from your terminal, with schedule-reporting and visualisation options.
Documentation for shadow is available at Read the Docs, or in the docs/
directory.
shadow is being actively developed by Ryan Bunney, a PhD Candidate at the International Centre for Radio Astronomy Research (ICRAR), in Perth, Western Australia.
shadow is built heavily on the following libraries;
- Networkx
- NumPy
Networkx forms the backbone of the Workflow objects that are used to interface with the scheduling algorithms. NumPy is used for efficient calculations and visualisation.