the-ai-summer/deep-learning-in-production

Build, train, deploy, scale and maintain deep learning models. Understand ML infrastructure and MLOps using hands-on examples.

Deep Learning In Production Book

You can know grab a copy of the book from here:

• Amazon: Paperback and Kindle
• Leanpub: Epub and Pdf

Build, train, deploy, scale and maintain deep learning models. Understand ML infrastructure and MLOps using hands-on examples.

What you will learn?

• Best practices to write Deep Learning code
• How to unit test and debug Machine Learning code
• How to build and deploy efficient data pipelines
• How to serve Deep Learning models
• How to deploy and scale your application
• What is MLOps and how to build end-to-end pipelines

Who is this book for?

• Software engineers who are starting out with deep learning
• Machine learning researchers with limited software engineering background
• Machine learning engineers who seek to strengthen their knowledge
• Data scientists who want to productionize their models and build customer-facing applications

What tools you will use?

Tensorflow, Flask, uWSGI, Nginx, Docker, Kubernetes, Tensorflow Extended, Google Cloud, Vertex AI

Book description

Deep Learning research is advancing rapidly over the past years. Frameworks and libraries are constantly been developed and updated. However, we still lack standardized solutions on how to serve, deploy and scale Deep Learning models. Deep Learning infrastructure is not very mature yet.

This book accumulates a set of best practices and approaches on how to build robust and scalable machine learning applications. It covers the entire lifecycle from data processing and training to deployment and maintenance. It will help you understand how to transfer methodologies that are generally accepted and applied in the software community, into Deep Learning projects.

It's an excellent choice for researchers with a minimal software background, software engineers with little experience in machine learning, or aspiring machine learning engineers.

More details and a free sample

Visit the book's page

Table of Contents

1. Designing a machine learning system
2. Setting up a Deep Learning Workstation
3. Writing and Structuring Deep Learning Code
4. Data Processing
5. Training
6. Serving
7. Deploying
8. Scaling
9. Building an End-to-End Pipeline

Articles

The books is based on an article series published in our blog "AI Summer" and they were later combined and organized into a single resource. Some were rewritten from scratch; some were modified to fit the book's structure. Plus, we added completely new material!

1. Laptop set up and system design: https://theaisummer.com/deep-learning-production/
2. Best practices to write Deep Learning code: Project structure, OOP, Type checking and documentation: https://theaisummer.com/best-practices-deep-learning-code/
3. How to Unit Test Deep Learning: Tests in TensorFlow, mocking and test coverage: https://theaisummer.com/unit-test-deep-learning/
4. Logging and Debugging in Machine Learning: https://theaisummer.com/logging-debugging/
5. Data preprocessing for deep learning: https://theaisummer.com/data-preprocessing/
6. Data preprocessing for deep learning (part2): https://theaisummer.com/data-processing-optimization/
7. How to build a custom production-ready Deep Learning Training loop in Tensorflow from scratch: https://theaisummer.com/tensorflow-training-loop/
8. How to train a deep learning model in the cloud: https://theaisummer.com/training-cloud/
9. Distributed Deep Learning training: Model and Data Parallelism in Tensorflow: https://theaisummer.com/distributed-training/
10. Deploy a Deep Learning model as a web application using Flask and Tensorflow: https://theaisummer.com/deploy-flask-tensorflow/
11. How to use uWSGI and Nginx to serve a Deep Learning model: https://theaisummer.com/uwsgi-nginx/
12. How to use Docker containers and Docker Compose for Deep Learning applications: https://theaisummer.com/docker/
13. Scalability in Machine Learning: Grow your model to serve millions of users: https://theaisummer.com/scalability/
14. Introduction to Kubernetes with Google Cloud: Deploy your Deep Learning model effortlessly: https://theaisummer.com/kubernetes/

Support

If you like our effort, don't forget to star the project :) It matters!

deep-learning-in-production's Issues

Dataloader not being used in the model.

It seems like the data loader is not being used in the model.py. Specificially, the preprocess_train and preprecess_data have been defined again in the model.py. I would appreciate little clarification

https://github.com/The-AI-Summer/Deep-Learning-In-Production/blob/master/10.%20How%20to%20use%20Docker%20containers%20and%20Docker%20Compose%20for%20Deep%20Learning%20applications/dataloader/dataloader.py#L11

<Response [200]> with Flask Application

Hello,

Thanks for putting together such a nice and detailed tutorial on DL in production, I have learned a lot and it is clear you put a lot of effort in making everything concrete and clear.

I am now trying to solve an issue with the simple Flask application tutorial. After writing all the code, I always get a <Response [200]> error at the client side. I have debugged everything in the server and client side and all seems to be working: the input image from the client is correctly read and preprocessed, the network outputs correct values, but when the client gets a response I always get that error. The only message that appears on the server side is 127.0.0.1 - - [28/Mar/2021 10:17:00] "POST /infer HTTP/1.1" 200 -

I have even tried return "hello" in my infer method (inferrer.py) and the infer method (service.py) to make sure the return value is json serializable, but to no avail.

Do you have any suggestions on how to continue debugging? Thank you.

Guide

Hi @SergiosKar @black0017 ,
How to run the code for demo?
Best regards,
PeterPham

Issue

Hi @black0017 @SergiosKar
When i run this code from the article "Deploy a Deep Learning model as a web application using Flask and Tensorflow". I have a issue:
ModuleNotFoundError: No module named 'tensorflow_examples'
What should i do?
Best regards,
PeterPham

Issues with the function: "dummy_load_data" of the module model.tests.unet_test

I have an issue with the following functions. "dummy_load_data" loads the dummy dataset.

#function of the module: model.tests.unet_test
  def dummy_load_data(*args, **kwargs):
      with tfds.testing.mock_data(num_examples=1):
          return tfds.load(CFG['data']['path'], with_info=True)

#function of the class: model.tests.unet_test.UnetTest
   @patch('model.unet.DataLoader.load_data')
    def test_load_data(self, mock_data_loader):
        mock_data_loader.side_effect = dummy_load_data
        shape = tf.TensorShape([None, self.unet.image_size, self.unet.image_size, 3])

        self.unet.load_data()
        mock_data_loader.assert_called()

        self.assertItemsEqual(self.unet.train_dataset.element_spec[0].shape, shape)
        self.assertItemsEqual(self.unet.test_dataset.element_spec[0].shape, shape)

#functions (which are supposed to be tested with test_load_data function above) "load_data" and  "_preprocess_data" of the class: model.unet.Unet 
    def load_data(self):
            """Loads and Preprocess data """
            self.dataset, self.info = DataLoader().load_data(self.config.data)
            self._preprocess_data()
    
    def _preprocess_data(self):
            """ Splits into training and test and set training parameters"""
            train = self.dataset['train'].map(self._load_image_train, num_parallel_calls=tf.data.experimental.AUTOTUNE)
            test = self.dataset['test'].map(self._load_image_test)
    
            self.train_dataset = train.cache().shuffle(self.buffer_size).batch(self.batch_size).repeat()
            self.train_dataset = self.train_dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)
            self.test_dataset = test.batch(self.batch_size)

I want to test the load_data function of the class model.unet by using a dummy data , which is loaded with the function
dummy_load_data. (above)

When I´m testing the function "load_data" of the class model.unet, I get the following error:
train = self.dataset["train"].map(map_func =self._load_image_train, num_parallel_calls=tf.data.AUTOTUNE) KeyError: 'train'

I think its because the dummy_load_data function just loads one sample. So the returned dataset of the dummy_load_data function doesnt have the keys ["train"] and ["test"]. Therefore its not possible to access these keys in the _preprocess_data function.

But how can I create a mock dataset with the keys ["train"] and ["test"]?