marco-piazzi/mlops

Modular and minimalistic MLOps recipes

MLOps Recipes

A library of minimalistic & modular recipes. Easily compose your custom end to end CI/CD pipelines for Machine Learning

Check out the Project website for more details

Status: Project board

Technologies: Azure Machine Learning & Azure Devops

Get Started

1. Check out the Project website
2. Setup the environment
3. Run an end to end MLOps pipeline

Note: Automated builds based on code/asset changes have been disabled by setting triggers: none in the pipelines. The reason is to avoid triggering accidental builds during your learning phase.

MLOps Flow

Notes on our Base scenario:

1. Directory Structure
   1. mlops contains the devops pipelines
      1. model_pipelines contains individual pipelines for each of the models. These leverage the recipes
      2. recipes: contains parameterized, reusable devops pipeline for different scenarios
   2. models directory has the source code for the individual models (training, scoring etc)
   3. setup directory contains documentation on usage
2. Training: For training we use a simple LogisticRegression model on the German Credit card dataset. We build sklearn pipeline that does feature engineering. We export the whole pipeline as a the model binary (pkl file).
3. We use Azure ML CLI as a mechanism for interacting with Azure ML due to simplicity reasons.

Acknowledgements

1. MLOpsPython python repo was one of the inspirations for this - thanks to the contributors
2. German Creditcard Dataset
   Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.