hp2706/ku_devops

MLOps for data scientists

This repository contains a small introduction to machine learning operations (MLOps) for data scientists. The four core topics covered are:

• Virtual environments
• Version control
• Experiment tracking
• Code testing

You are supposed to do them in the order listed. When doing the exercises, do maximize your MLOps/DevOps experience you should priorities:

1. Make yourself familiar with running commands in the terminal. The terminal can be a scary place, but it is an essential skill to be able to run commands without relying on a graphical interface. If you want good introduction to using the shell, I highly recommend the first two lectures from this MIT course.
2. Only use scripts e.g. no notebooks for these exercises. Notebooks have their benefits but the fact is that developing software in the real world is done in scripts. Therefore make sure that whenever you are writing code for the exercises do this in .py scripts. If you feel like you miss the interactiveness of notebooks when working with script I can highly recommend giving ipython a spin.
3. Get a good code editor, and try using it. If you do not have one, I can highly recommend Visual Studio Code that are a lightweight editor, but through extensions can become really powerfull. Otherwise, I also recommend PyCharm.

Why should a data scientist care about MLOps? Because MLOps provide processes and tools for creating reproducible experiments at scale when working with any kind of machine learning models. Being able secure that your experiments are reproducible are important in the context of the scientific method:

Image credit

Without reproducibility the method breaks at the experimental stage, as non-reproducible experiments will most likely lead to different results and thereby different conclusions on the initial hypothesis.

For a much more complete set of material on this topic, see this course which goes over the near complete pipeline of designing, modeling and deploying machine learning applications.