mrdbourke/tensorflow-deep-learning

All course materials for the Zero to Mastery Deep Learning with TensorFlow course.

Zero to Mastery Deep Learning with TensorFlow

All of the course materials for the Zero to Mastery Deep Learning with TensorFlow course.

This course will teach you the foundations of deep learning and TensorFlow as well as prepare you to pass the TensorFlow Developer Certification exam (optional).

Important links

• 🎥 Watch the first 14-hours of the course on YouTube (notebooks 00, 01, 02)
• 📖 Read the beautiful online book version of the course
• 💻 Sign up to the full course on the Zero to Mastery Academy (videos for notebooks 03-10)
• 🤔 Got questions about the course? Check out the livestream Q&A for the course launch
• 📝 Get a quick overview of TensorFlow with the TensorFlow Cheatsheet

Contents of this page

• Fixes and updates
• Course materials (everything you'll need for completing the course)
• Course structure (how this course is taught)
• Should you do this course? (decide by answering a couple simple questions)
• Prerequisites (what skills you'll need to do this course)
• Exercises & Extra-curriculum (challenges to practice what you've learned and resources to learn more)
• Ask a question (like to know more? go here)
• Status
• Log (updates, changes and progress)

Fixes and updates

• 18 Aug 2023 - Update Notebook 05 to fix #544 and #553, see #575 for full notes
  • In short, if you're using tf.keras.applications.EfficientNetB0 and facing errors, swap to tf.keras.applications.efficientnet_v2.EfficientNetV2B0
• 26 May 2023 - Update Notebook 08 for new version of TensorFlow + update Notebook 09 for new version of TensorFlow & spaCy, see update notes for 09: #557
• 19 May 2023 - Update Notebook 07 for new version of TensorFlow + fix model loading errors (TensorFlow 2.13+ required), see: #550
• 18 May 2023 - Update Notebook 06 for new TensorFlow namespaces (no major functionality change, just different imports), see: #549
• 12 May 2023 - Notebook 05 new namespaces added for tf.keras.layers, see #547, also add fix for issue with model.load_weights() in Notebook 05, see #544, if you're having trouble saving/loading the model weights, also see #553
• 12 May 2023 - Newer versions of TensorFlow (2.10+) use learning_rate instead of lr in tf.keras.optimizers (e.g. tf.keras.optimizers.Adam(learning_rate=0.001), old lr still works but is deprecated
• 02 Dec 2021 - Added fix for TensorFlow 2.7.0+ for notebook 02, see discussion for more
• 11 Nov 2021 - Added fix for TensorFlow 2.7.0+ for notebook 01, see discussion for more

Course materials

This table is the ground truth for course materials. All the links you need for everything will be here.

Key:

• Number: The number of the target notebook (this may not match the video section of the course but it ties together all of the materials in the table)
• Notebook: The notebook for a particular module with lots of code and text annotations (notebooks from the videos are based on these)
• Data/model: Links to datasets/pre-trained models for the associated notebook
• Exercises & Extra-curriculum: Each module comes with a set of exercises and extra-curriculum to help practice your skills and learn more, I suggest going through these before you move onto the next module
• Slides: Although we focus on writing TensorFlow code, we sometimes use pretty slides to describe different concepts, you'll find them here

Note: You can get all of the notebook code created during the videos in the video_notebooks directory.

Number	Notebook	Data/Model	Exercises & Extra-curriculum	Slides
00	TensorFlow Fundamentals		Go to exercises & extra-curriculum	Go to slides
01	TensorFlow Regression		Go to exercises & extra-curriculum	Go to slides
02	TensorFlow Classification		Go to exercises & extra-curriculum	Go to slides
03	TensorFlow Computer Vision	pizza_steak, 10_food_classes_all_data	Go to exercises & extra-curriculum	Go to slides
04	Transfer Learning Part 1: Feature extraction	10_food_classes_10_percent	Go to exercises & extra-curriculum	Go to slides
05	Transfer Learning Part 2: Fine-tuning	10_food_classes_10_percent, 10_food_classes_1_percent, 10_food_classes_all_data	Go to exercises & extra-curriculum	Go to slides
06	Transfer Learning Part 3: Scaling up	101_food_classes_10_percent, custom_food_images, fine_tuned_efficientnet_model	Go to exercises & extra-curriculum	Go to slides
07	Milestone Project 1: Food Vision 🍔👁, Template (your challenge)	feature_extraction_mixed_precision_efficientnet_model, fine_tuned_mixed_precision_efficientnet_model	Go to exercises & extra-curriculum	Go to slides
08	TensorFlow NLP Fundamentals	diaster_or_no_diaster_tweets, USE_feature_extractor_model	Go to exercises & extra-curriculum	Go to slides
09	Milestone Project 2: SkimLit 📄🔥	pubmed_RCT_200k_dataset, skimlit_tribrid_model	Go to exercises & extra-curriculum	Go to slides
10	TensorFlow Time Series Fundamentals & Milestone Project 3: BitPredict 💰📈	bitcoin_price_data_USD_2013-10-01_2021-05-18.csv	Go to exercises & extra-curriculum	Go to slides
11	Preparing to Pass the TensorFlow Developer Certification Exam		Go to exercises & extra-curriculum	Go to slides

Course structure

This course is code first. The goal is to get you writing deep learning code as soon as possible.

It is taught with the following mantra:

Code -> Concept -> Code -> Concept -> Code -> Concept

This means we write code first then step through the concepts behind it.

If you've got 6-months experience writing Python code and a willingness to learn (most important), you'll be able to do the course.

Should you do this course?

Do you have 1+ years experience with deep learning and writing TensorFlow code?

If yes, no you shouldn't, use your skills to build something.

If no, move onto the next question.

Have you done at least one beginner machine learning course and would like to learn about deep learning/pass the TensorFlow Developer Certification?

If yes, this course is for you.

If no, go and do a beginner machine learning course and if you decide you want to learn TensorFlow, this page will still be here.

Prerequisites

What do I need to know to go through this course?

• 6+ months writing Python code. Can you write a Python function which accepts and uses parameters? That’s good enough. If you don’t know what that means, spend another month or two writing Python code and then come back here.
• At least one beginner machine learning course. Are you familiar with the idea of training, validation and test sets? Do you know what supervised learning is? Have you used pandas, NumPy or Matplotlib before? If no to any of these, I’d going through at least one machine learning course which teaches these first and then coming back.
• Comfortable using Google Colab/Jupyter Notebooks. This course uses Google Colab throughout. If you have never used Google Colab before, it works very similar to Jupyter Notebooks with a few extra features. If you’re not familiar with Google Colab notebooks, I’d suggest going through the Introduction to Google Colab notebook.
• Plug: The Zero to Mastery beginner-friendly machine learning course (I also teach this) teaches all of the above (and this course is designed as a follow on).

🛠 Exercises & 📖 Extra-curriculum

To prevent the course from being 100+ hours (deep learning is a broad field), various external resources for different sections are recommended to puruse under your own discretion.

You can find solutions to the exercises in extras/solutions/, there's a notebook per set of exercises (one for 00, 01, 02... etc). Thank you to Ashik Shafi for all of the efforts creating these.

---

🛠 00. TensorFlow Fundamentals Exercises

1. Create a vector, scalar, matrix and tensor with values of your choosing using tf.constant().
2. Find the shape, rank and size of the tensors you created in 1.
3. Create two tensors containing random values between 0 and 1 with shape [5, 300].
4. Multiply the two tensors you created in 3 using matrix multiplication.
5. Multiply the two tensors you created in 3 using dot product.
6. Create a tensor with random values between 0 and 1 with shape [224, 224, 3].
7. Find the min and max values of the tensor you created in 6 along the first axis.
8. Created a tensor with random values of shape [1, 224, 224, 3] then squeeze it to change the shape to [224, 224, 3].
9. Create a tensor with shape [10] using your own choice of values, then find the index which has the maximum value.
10. One-hot encode the tensor you created in 9.

📖 00. TensorFlow Fundamentals Extra-curriculum

• Read through the list of TensorFlow Python APIs, pick one we haven't gone through in this notebook, reverse engineer it (write out the documentation code for yourself) and figure out what it does.
• Try to create a series of tensor functions to calculate your most recent grocery bill (it's okay if you don't use the names of the items, just the price in numerical form).
  • How would you calculate your grocery bill for the month and for the year using tensors?
• Go through the TensorFlow 2.x quick start for beginners tutorial (be sure to type out all of the code yourself, even if you don't understand it).
  • Are there any functions we used in here that match what's used in there? Which are the same? Which haven't you seen before?
• Watch the video "What's a tensor?" - a great visual introduction to many of the concepts we've covered in this notebook.

---

🛠 01. Neural network regression with TensorFlow Exercises

1. Create your own regression dataset (or make the one we created in "Create data to view and fit" bigger) and build fit a model to it.
2. Try building a neural network with 4 Dense layers and fitting it to your own regression dataset, how does it perform?
3. Try and improve the results we got on the insurance dataset, some things you might want to try include:

• Building a larger model (how does one with 4 dense layers go?).
• Increasing the number of units in each layer.
• Lookup the documentation of Adam and find out what the first parameter is, what happens if you increase it by 10x?
• What happens if you train for longer (say 300 epochs instead of 200)?

4. Import the Boston pricing dataset from TensorFlow tf.keras.datasets and model it.

📖 01. Neural network regression with TensorFlow Extra-curriculum

• MIT introduction deep learning lecture 1 - gives a great overview of what's happening behind all of the code we're running.
• Reading: 1-hour of Chapter 1 of Neural Networks and Deep Learning by Michael Nielson - a great in-depth and hands-on example of the intuition behind neural networks.
• To practice your regression modelling with TensorFlow, I'd also encourage you to look through Kaggle's datasets, find a regression dataset which sparks your interest and try to model.

---

🛠 02. Neural network classification with TensorFlow Exercises

1. Play with neural networks in the TensorFlow Playground for 10-minutes. Especially try different values of the learning, what happens when you decrease it? What happens when you increase it?
2. Replicate the model pictured in the TensorFlow Playground diagram below using TensorFlow code. Compile it using the Adam optimizer, binary crossentropy loss and accuracy metric. Once it's compiled check a summary of the model. Try this network out for yourself on the TensorFlow Playground website. Hint: there are 5 hidden layers but the output layer isn't pictured, you'll have to decide what the output layer should be based on the input data.
3. Create a classification dataset using Scikit-Learn's make_moons() function, visualize it and then build a model to fit it at over 85% accuracy.
4. Train a model to get 88%+ accuracy on the fashion MNIST test set. Plot a confusion matrix to see the results after.
5. Recreate TensorFlow's softmax activation function in your own code. Make sure it can accept a tensor and return that tensor after having the softmax function applied to it.
6. Create a function (or write code) to visualize multiple image predictions for the fashion MNIST at the same time. Plot at least three different images and their prediction labels at the same time. Hint: see the classification tutorial in the TensorFlow documentation for ideas.
7. Make a function to show an image of a certain class of the fashion MNIST dataset and make a prediction on it. For example, plot 3 images of the T-shirt class with their predictions.

📖 02. Neural network classification with TensorFlow Extra-curriculum

• Watch 3Blue1Brown's neural networks video 2: Gradient descent, how neural networks learn. After you're done, write 100 words about what you've learned.
  • If you haven't already, watch video 1: But what is a Neural Network?. Note the activation function they talk about at the end.
• Watch MIT's introduction to deep learning lecture 1 (if you haven't already) to get an idea of the concepts behind using linear and non-linear functions.
• Spend 1-hour reading Michael Nielsen's Neural Networks and Deep Learning book.
• Read the ML-Glossary documentation on activation functions. Which one is your favourite?
  • After you've read the ML-Glossary, see which activation functions are available in TensorFlow by searching "tensorflow activation functions".

---

🛠 03. Computer vision & convolutional neural networks in TensorFlow Exercises

1. Spend 20-minutes reading and interacting with the CNN explainer website.

• What are the key terms? e.g. explain convolution in your own words, pooling in your own words

2. Play around with the "understanding hyperparameters" section in the CNN explainer website for 10-minutes.

• What is the kernel size?
• What is the stride?
• How could you adjust each of these in TensorFlow code?

3. Take 10 photos of two different things and build your own CNN image classifier using the techniques we've built here.
4. Find an ideal learning rate for a simple convolutional neural network model on your the 10 class dataset.

📖 03. Computer vision & convolutional neural networks in TensorFlow Extra-curriculum

• Watch: MIT's Introduction to Deep Computer Vision lecture. This will give you a great intuition behind convolutional neural networks.
• Watch: Deep dive on mini-batch gradient descent by deeplearning.ai. If you're still curious about why we use batches to train models, this technical overview covers many of the reasons why.
• Read: CS231n Convolutional Neural Networks for Visual Recognition class notes. This will give a very deep understanding of what's going on behind the scenes of the convolutional neural network architectures we're writing.
• Read: "A guide to convolution arithmetic for deep learning". This paper goes through all of the mathematics running behind the scenes of our convolutional layers.
• Code practice: TensorFlow Data Augmentation Tutorial. For a more in-depth introduction on data augmentation with TensorFlow, spend an hour or two reading through this tutorial.

---

🛠 04. Transfer Learning in TensorFlow Part 1: Feature Extraction Exercises

1. Build and fit a model using the same data we have here but with the MobileNetV2 architecture feature extraction (mobilenet_v2_100_224/feature_vector) from TensorFlow Hub, how does it perform compared to our other models?
2. Name 3 different image classification models on TensorFlow Hub that we haven't used.
3. Build a model to classify images of two different things you've taken photos of.

• You can use any feature extraction layer from TensorFlow Hub you like for this.
• You should aim to have at least 10 images of each class, for example to build a fridge versus oven classifier, you'll want 10 images of fridges and 10 images of ovens.

4. What is the current best performing model on ImageNet?

• Hint: you might want to check sotabench.com for this.

📖 04. Transfer Learning in TensorFlow Part 1: Feature Extraction Extra-curriculum

• Read through the TensorFlow Transfer Learning Guide and define the main two types of transfer learning in your own words.
• Go through the Transfer Learning with TensorFlow Hub tutorial on the TensorFlow website and rewrite all of the code yourself into a new Google Colab notebook making comments about what each step does along the way.
• We haven't covered fine-tuning with TensorFlow Hub in this notebook, but if you'd like to know more, go through the fine-tuning a TensorFlow Hub model tutorial on the TensorFlow homepage.How to fine-tune a tensorflow hub model:
• Look into experiment tracking with Weights & Biases, how could you integrate it with our existing TensorBoard logs?

---

🛠 05. Transfer Learning in TensorFlow Part 2: Fine-tuning Exercises

1. Use feature-extraction to train a transfer learning model on 10% of the Food Vision data for 10 epochs using tf.keras.applications.EfficientNetB0 as the base model. Use the ModelCheckpoint callback to save the weights to file.
2. Fine-tune the last 20 layers of the base model you trained in 2 for another 10 epochs. How did it go?
3. Fine-tune the last 30 layers of the base model you trained in 2 for another 10 epochs. How did it go?
4. Write a function to visualize an image from any dataset (train or test file) and any class (e.g. "steak", "pizza"... etc), visualize it and make a prediction on it using a trained model.

📖 05. Transfer Learning in TensorFlow Part 2: Fine-tuning Extra-curriculum

• Read the documentation on data augmentation in TensorFlow.
• Read the ULMFit paper (technical) for an introduction to the concept of freezing and unfreezing different layers.
• Read up on learning rate scheduling (there's a TensorFlow callback for this), how could this influence our model training?
  • If you're training for longer, you probably want to reduce the learning rate as you go... the closer you get to the bottom of the hill, the smaller steps you want to take. Imagine it like finding a coin at the bottom of your couch. In the beginning your arm movements are going to be large and the closer you get, the smaller your movements become.

---

🛠 06. Transfer Learning in TensorFlow Part 3: Scaling-up Exercises

1. Take 3 of your own photos of food and use the trained model to make predictions on them, share your predictions with the other students in Discord and show off your Food Vision model 🍔👁.
2. Train a feature-extraction transfer learning model for 10 epochs on the same data and compare its performance versus a model which used feature extraction for 5 epochs and fine-tuning for 5 epochs (like we've used in this notebook). Which method is better?
3. Recreate the first model (the feature extraction model) with mixed_precision turned on.

• Does it make the model train faster?
• Does it effect the accuracy or performance of our model?
• What's the advantages of using mixed_precision training?

📖 06. Transfer Learning in TensorFlow Part 3: Scaling-up Extra-curriculum

• Spend 15-minutes reading up on the EarlyStopping callback. What does it do? How could we use it in our model training?
• Spend an hour reading about Streamlit. What does it do? How might you integrate some of the things we've done in this notebook in a Streamlit app?

---

🛠 07. Milestone Project 1: 🍔👁 Food Vision Big™ Exercises

Note: The chief exercise for Milestone Project 1 is to finish the "TODO" sections in the Milestone Project 1 Template notebook. After doing so, move onto the following.

1. Use the same evaluation techniques on the large-scale Food Vision model as you did in the previous notebook (Transfer Learning Part 3: Scaling up). More specifically, it would be good to see:

• A confusion matrix between all of the model's predictions and true labels.
• A graph showing the f1-scores of each class.
• A visualization of the model making predictions on various images and comparing the predictions to the ground truth.
  • For example, plot a sample image from the test dataset and have the title of the plot show the prediction, the prediction probability and the ground truth label.
• Note: To compare predicted labels to test labels, it might be a good idea when loading the test data to set shuffle=False (so the ordering of test data is preserved alongside the order of predicted labels).

2. Take 3 of your own photos of food and use the Food Vision model to make predictions on them. How does it go? Share your images/predictions with the other students.
3. Retrain the model (feature extraction and fine-tuning) we trained in this notebook, except this time use EfficientNetB4 as the base model instead of EfficientNetB0. Do you notice an improvement in performance? Does it take longer to train? Are there any tradeoffs to consider?
4. Name one important benefit of mixed precision training, how does this benefit take place?

📖 07. Milestone Project 1: 🍔👁 Food Vision Big™ Extra-curriculum

• Read up on learning rate scheduling and the learning rate scheduler callback. What is it? And how might it be helpful to this project?
• Read up on TensorFlow data loaders (improving TensorFlow data loading performance). Is there anything we've missed? What methods you keep in mind whenever loading data in TensorFlow? Hint: check the summary at the bottom of the page for a great round up of ideas.
• Read up on the documentation for TensorFlow mixed precision training. What are the important things to keep in mind when using mixed precision training?

---

🛠 08. Introduction to NLP (Natural Language Processing) in TensorFlow Exercises

1. Rebuild, compile and train model_1, model_2 and model_5 using the Keras Sequential API instead of the Functional API.
2. Retrain the baseline model with 10% of the training data. How does perform compared to the Universal Sentence Encoder model with 10% of the training data?
3. Try fine-tuning the TF Hub Universal Sentence Encoder model by setting training=True when instantiating it as a Keras layer.

# We can use this encoding layer in place of our text_vectorizer and embedding layer
sentence_encoder_layer = hub.KerasLayer("https://tfhub.dev/google/universal-sentence-encoder/4",
                                        input_shape=[],
                                        dtype=tf.string,
                                        trainable=True) # turn training on to fine-tune the TensorFlow Hub model

4. Retrain the best model you've got so far on the whole training set (no validation split). Then use this trained model to make predictions on the test dataset and format the predictions into the same format as the sample_submission.csv file from Kaggle (see the Files tab in Colab for what the sample_submission.csv file looks like). Once you've done this, make a submission to the Kaggle competition, how did your model perform?
5. Combine the ensemble predictions using the majority vote (mode), how does this perform compare to averaging the prediction probabilities of each model?
6. Make a confusion matrix with the best performing model's predictions on the validation set and the validation ground truth labels.

📖 08. Introduction to NLP (Natural Language Processing) in TensorFlow Extra-curriculum

To practice what you've learned, a good idea would be to spend an hour on 3 of the following (3-hours total, you could through them all if you want) and then write a blog post about what you've learned.

• For an overview of the different problems within NLP and how to solve them read through:
  • A Simple Introduction to Natural Language Processing
  • How to solve 90% of NLP problems: a step-by-step guide
• Go through MIT's Recurrent Neural Networks lecture. This will be one of the greatest additions to what's happening behind the RNN model's you've been building.
• Read through the word embeddings page on the TensorFlow website. Embeddings are such a large part of NLP. We've covered them throughout this notebook but extra practice would be well worth it. A good exercise would be to write out all the code in the guide in a new notebook.
• For more on RNN's in TensorFlow, read and reproduce the TensorFlow RNN guide. We've covered many of the concepts in this guide, but it's worth writing the code again for yourself.
• Text data doesn't always come in a nice package like the data we've downloaded. So if you're after more on preparing different text sources for being with your TensorFlow deep learning models, it's worth checking out the following:
  • TensorFlow text loading tutorial.
  • Reading text files with Python by Real Python.
• This notebook has focused on writing NLP code. For a mathematically rich overview of how NLP with Deep Learning happens, read Stanford's Natural Language Processing with Deep Learning lecture notes Part 1.
  • For an even deeper dive, you could even do the whole CS224n (Natural Language Processing with Deep Learning) course.
• Great blog posts to read:
  • Andrei Karpathy's The Unreasonable Effectiveness of RNNs dives into generating Shakespeare text with RNNs.
  • Text Classification with NLP: Tf-Idf vs Word2Vec vs BERT by Mauro Di Pietro. An overview of different techniques for turning text into numbers and then classifying it.
  • What are word embeddings? by Machine Learning Mastery.
• Other topics worth looking into:
  • Attention mechanisms. These are a foundational component of the transformer architecture and also often add improvements to deep NLP models.
  • Transformer architectures. This model architecture has recently taken the NLP world by storm, achieving state of the art on many benchmarks. However, it does take a little more processing to get off the ground, the HuggingFace Models (formerly HuggingFace Transformers) library is probably your best quick start.

---

🛠 09. Milestone Project 2: SkimLit 📄🔥 Exercises

1. Train model_5 on all of the data in the training dataset for as many epochs until it stops improving. Since this might take a while, you might want to use:

• tf.keras.callbacks.ModelCheckpoint to save the model's best weights only.
• tf.keras.callbacks.EarlyStopping to stop the model from training once the validation loss has stopped improving for ~3 epochs.

2. Checkout the Keras guide on using pretrained GloVe embeddings. Can you get this working with one of our models?

• Hint: You'll want to incorporate it with a custom token Embedding layer.
• It's up to you whether or not you fine-tune the GloVe embeddings or leave them frozen.

3. Try replacing the TensorFlow Hub Universal Sentence Encoder pretrained embedding for the TensorFlow Hub BERT PubMed expert (a language model pretrained on PubMed texts) pretrained embedding. Does this effect results?

• Note: Using the BERT PubMed expert pretrained embedding requires an extra preprocessing step for sequences (as detailed in the TensorFlow Hub guide).
• Does the BERT model beat the results mentioned in this paper? https://arxiv.org/pdf/1710.06071.pdf

4. What happens if you were to merge our line_number and total_lines features for each sequence? For example, created a X_of_Y feature instead? Does this effect model performance?

• Another example: line_number=1 and total_lines=11 turns into line_of_X=1_of_11.

5. Write a function (or series of functions) to take a sample abstract string, preprocess it (in the same way our model has been trained), make a prediction on each sequence in the abstract and return the abstract in the format:

• PREDICTED_LABEL: SEQUENCE
• PREDICTED_LABEL: SEQUENCE
• PREDICTED_LABEL: SEQUENCE
• PREDICTED_LABEL: SEQUENCE
• ...
  • You can find your own unstructured RCT abstract from PubMed or try this one from: Baclofen promotes alcohol abstinence in alcohol dependent cirrhotic patients with hepatitis C virus (HCV) infection.

📖 09. Milestone Project 2: SkimLit 📄🔥 Extra-curriculum

• For more on working with text/spaCy, see spaCy's advanced NLP course. If you're going to be working on production-level NLP problems, you'll probably end up using spaCy.
• For another look at how to approach a text classification problem like the one we've just gone through, I'd suggest going through Google's Machine Learning Course for text classification.
• Since our dataset has imbalanced classes (as with many real-world datasets), so it might be worth looking into the TensorFlow guide for different methods to training a model with imbalanced classes.

---

🛠 10. Time series fundamentals and Milestone Project 3: BitPredict 💰📈 Exercises

1. Does scaling the data help for univariate/multivariate data? (e.g. getting all of the values between 0 & 1)

• Try doing this for a univariate model (e.g. model_1) and a multivariate model (e.g. model_6) and see if it effects model training or evaluation results.

2. Get the most up to date data on Bitcoin, train a model & see how it goes (our data goes up to May 18 2021).

• You can download the Bitcoin historical data for free from coindesk.com/price/bitcoin and clicking "Export Data" -> "CSV".

3. For most of our models we used WINDOW_SIZE=7, but is there a better window size?

• Setup a series of experiments to find whether or not there's a better window size.
• For example, you might train 10 different models with HORIZON=1 but with window sizes ranging from 2-12.

4. Create a windowed dataset just like the ones we used for model_1 using tf.keras.preprocessing.timeseries_dataset_from_array() and retrain model_1 using the recreated dataset.
5. For our multivariate modelling experiment, we added the Bitcoin block reward size as an extra feature to make our time series multivariate.

• Are there any other features you think you could add?
• If so, try it out, how do these affect the model?

6. Make prediction intervals for future forecasts. To do so, one way would be to train an ensemble model on all of the data, make future forecasts with it and calculate the prediction intervals of the ensemble just like we did for model_8.
7. For future predictions, try to make a prediction, retrain a model on the predictions, make a prediction, retrain a model, make a prediction, retrain a model, make a prediction (retrain a model each time a new prediction is made). Plot the results, how do they look compared to the future predictions where a model wasn't retrained for every forecast (model_9)?
8. Throughout this notebook, we've only tried algorithms we've handcrafted ourselves. But it's worth seeing how a purpose built forecasting algorithm goes.

• Try out one of the extra algorithms listed in the modelling experiments part such as:
  • Facebook's Kats library - there are many models in here, remember the machine learning practioner's motto: experiment, experiment, experiment.
  • LinkedIn's Greykite library

📖 10. Time series fundamentals and Milestone Project 3: BitPredict 💰📈 Extra-curriculum

We've only really scratched the surface with time series forecasting and time series modelling in general. But the good news is, you've got plenty of hands-on coding experience with it already.

If you'd like to dig deeper in to the world of time series, I'd recommend the following:

• Forecasting: Principles and Practice is an outstanding online textbook which discusses at length many of the most important concepts in time series forecasting. I'd especially recommend reading at least Chapter 1 in full.
  • I'd definitely recommend at least checking out chapter 1 as well as the chapter on forecasting accuracy measures.
• 🎥 Introduction to machine learning and time series by Markus Loning goes through different time series problems and how to approach them. It focuses on using the sktime library (Scikit-Learn for time series), though the principles are applicable elsewhere.
• Why you should care about the Nate Silver vs. Nassim Taleb Twitter war by Isaac Faber is an outstanding discussion insight into the role of uncertainty in the example of election prediction.
• TensorFlow time series tutorial - A tutorial on using TensorFlow to forecast weather time series data with TensorFlow.
• 📕 The Black Swan by Nassim Nicholas Taleb - Nassim Taleb was a pit trader (a trader who trades on their own behalf) for 25 years, this book compiles many of the lessons he learned from first-hand experience. It changed my whole perspective on our ability to predict.
• 3 facts about time series forecasting that surprise experienced machine learning practitioners by Skander Hannachi, Ph.D - time series data is different to other kinds of data, if you've worked on other kinds of machine learning problems before, getting into time series might require some adjustments, Hannachi outlines 3 of the most common.
• 🎥 World-class lectures by Jordan Kern, watching these will take you from 0 to 1 with time series problems:
  • Time Series Analysis - how to analyse time series data.
  • Time Series Modelling - different techniques for modelling time series data (many of which aren't deep learning).

---

🛠 11. Passing the TensorFlow Developer Certification Exercises

Preparing your brain

1. Read through the TensorFlow Developer Certificate Candidate Handbook.
2. Go through the Skills checklist section of the TensorFlow Developer Certification Candidate Handbook and create a notebook which covers all of the skills required, write code for each of these (this notebook can be used as a point of reference during the exam).

Example of mapping the Skills checklist section of the TensorFlow Developer Certification Candidate handbook to a notebook.

Prearing your computer

1. Go through the PyCharm quick start tutorials to make sure you're familiar with PyCharm (the exam uses PyCharm, you can download the free version).
2. Read through and follow the suggested steps in the setting up for the TensorFlow Developer Certificate Exam guide.
3. After going through (2), go into PyCharm and make sure you can train a model in TensorFlow. The model and dataset in the example image_classification_test.py script on GitHub should be enough. If you can train and save the model in under 5-10 minutes, your computer will be powerful enough to train the models in the exam.
   • Make sure you've got experience running models locally in PyCharm before taking the exam. Google Colab (what we used through the course) is a little different to PyCharm.

Before taking the exam make sure you can run TensorFlow code on your local machine in PyCharm. If the example image_class_test.py script can run completely in under 5-10 minutes on your local machine, your local machine can handle the exam (if not, you can use Google Colab to train, save and download models to submit for the exam).

📖 11. Passing the TensorFlow Developer Certification Extra-curriculum

If you'd like some extra materials to go through to further your skills with TensorFlow and deep learning in general or to prepare more for the exam, I'd highly recommend the following:

• 📄 Read: How I got TensorFlow Developer Certified (and how you can too)
• 🎥 Watch: How I passed the TensorFlow Developer Certification exam (and how you can too)
• Go through the TensorFlow in Practice Specialization on Coursera
• Read through the second half of Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow 2nd Edition

What this course is missing

Deep learning is a broad topic. So this course doesn't cover it all.

Here are some of the main topics you might want to look into next:

• Transformers (the neural network architecture taking the NLP world by storm)
• Multi-modal models (models which use more than one data source such as text & images)
• Reinforcement learning
• Unsupervised learning

Extensions (possible places to go after the course)

• Neural Networks and Deep Learning Book by Michael Nielsen - If the Zero to Mastery TensorFlow for Deep Learning course is top down, this book is bottom up. A fantastic resource to sandwich your knowledge.
• Deeplearning.AI specializations - The ZTM TensorFLow course focuses on code-first, the deeplearning.ai specializations will teach you what's going on behind the code.
• Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow Book (especially the 2nd half) - Many of the materials in this course were inspired by and guided by the pages of this beautiful text book.
• Full Stack Deep Learning - Learn how to turn your models into machine learning-powered applications.
• Made with ML MLOps materials - Similar to Full Stack Deep Learning but comprised into many small lessons around all the pieces of the puzzle (data collection, labelling, deployment and more) required to build a full-stack machine learning-powered application.
• fast.ai Curriculum - One of the best (and free) AI/deep learning courses online. Enough said.
• "How does a beginner data scientist like me gain experience?" by Daniel Bourke - Read this on how to get experience for a job after studying online/at unveristy (start the job before you have it).

Ask questions

Contact Daniel Bourke or add a discussion (preferred).

Status

As of: 12 May 2023

Course completed! 🕺

Any further updates/changes will be added below.

Log

• 12 May 2023 - update several course notebooks for latest version of TensorFlow, several API updates for Notebook 05 here: #547
• 02 Dec 2021 - add fix for TensorFlow 2.7 to notebook 02
• 11 Nov 2021 - add fix for TensorFlow 2.7 to notebook 01
• 14 Aug 2021 - added a discussion with TensorFlow 2.6 updates and EfficientNetV2 notes: #166
• 16 Jul 2021 - added 35 videos to ZTM Academy + Udemy versions of the course for time series and how to pass TensorFlow Developer Certification
• 10 Jul 2021 - added 29 edited time series videos to ZTM Academy + Udemy versions of the course, more to come soon
• 07 Jul 2021 - recorded 5 videos for passing TensorFlow Developer Certification exam section - ALL VIDEOS FOR COURSE DONE!!! time to edit/upload! 🎉
• 06 Jul 2021 - added guide to TensorFlow Certification Exam: https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/11_passing_the_tensorflow_developer_certification_exam.md - going to record videos for it tomorrow
• 05 Jul 2021 - making materials for TF certification exam (what/why/how)
• 02 Jul 2021 - FINISHED RECORDING VIDEOS FOR TIME SERIES SECTION!!!!! time to upload
• 30 Jun 2021 - recorded 12 videos for time series section, total heading past 60 (the biggest section yet), nearly done!!!
• 29 Jun 2021 - recorded 10 videos for time series section, total heading towards 60
• 28 Jun 2021 - recorded 10 videos for time series section, the line below says 40 videos total, actually more like 50
• 26 Jun 2021 - recorded 4 videos for time series section, looks like it'll be about 40 videos total
• 25 Jun 2021 - recorded 8 videos for time series section + fixed a bunch of typos in time series notebook
• 24 Jun 2021 - recorded 14 videos for time series section, more to come tomorrow
• 23 Jun 2021 - finished adding images to time series notebook, now to start video recording
• 22 Jun 2021 - added a bunch of images to the time series notebook/started making slides
• 21 Jun 2021 - code for time series notebook is done, now creating slides/images to prepare for recording
• 19 Jun 2021 - turned curriculum into an online book, you can read it here: https://dev.mrdbourke.com/tensorflow-deep-learning/
• 18 Jun 2021 - add exercises/extra-curriculum/outline to time series notebook
• 17 Jun 2021 - add annotations for turkey problem and model comparison in time series notebook, next is outline/images
• 16 Jun 2021 - add annotations for uncertainty and future predictions in time series notebook, next is turkey problem
• 14 Jun 2021 - add annotations for ensembling, begin on prediction intervals
• 10 Jun 2021 - finished annotations for N-BEATS algorithm, now onto ensembling/prediction intervals
• 9 Jun 2021 - add annotations for N-BEATS algorithm implementation for time series notebook
• 8 Jun 2021 - add annotations to time series notebook, all will be finished by end of week (failed)
• 4 Jun 2021 - more annotation updates to time series notebook, brick by brick!
• 3 Jun 2021 - added a bunch of annotations/explanations to time series notebook, momentum building, plenty more to come!
• 2 Jun 2021 - started adding annotations explaining the code + resources to learn more, will continue for next few days
• 1 Jun 2021 - added turkey problem to time series notebook, cleaned up a bunch of code, draft code is ready, now to write annotations/explanations
• 28 May 2021 - added future forecasts, added ensemble model, added prediction intervals to time series notebook
• 25 May 2021 - added multivariate time series to time series notebook, fix LSTM model, next we add TensorFlow windowing/experimenting with window sizes
• 24 May 2021 - fixed broken preprocessing function in time series notebook, LSTM model is broken, more material to come
• 20 May 2021 - more time series material creation
• 19 May 2021 - more time series material creation, streaming much of it live on Twitch - https://twitch.tv/mrdbourke
• 18 May 2021 - added time series forecasting notebook outline (notebook 10), going to really start ramping up the materials here
• 12 May 2021 - all videos for 09 have now been released on Udemy & ZTM!!! enjoy build SkimLit 📄🔥
• 11 May 2021 - 40+ section 08 & 09 videos released on Udemy & ZTM!!!
• 10 May 2021 - time series materials research + preparation
• 08 May 2021 - time series materials research + preparation
• 05 May 2021 - ~20+ videos edited for 08, ~10+ videos edited for 09, time series materials in 1st draft mode
• 04 May 2021 - fixed the remaining videos for 08 (audio missing), now onto making time series materials!
• 03 May 2021 - rerecorded 10 videos for 08 fixing the sound isse, these are going straight to editing and should be uploaded by end of week
• 02 May 2021 - found an issue with videos 09-20 of section 08 (no audio), going to rerecord them
• 29 Apr 2021 - 🚀🚀🚀 launched on Udemy!!! 🚀🚀🚀
• 22 Apr 2021 - finished recording videos for 09! added slides and video notebook 09
• 21 Apr 2021 - recorded 14 videos for 09! biggggg day of recording! getting closer to finishing 09
• 20 Apr 2021 - recorded 10 videos for 09
• 19 Apr 2021 - recorded 9 videos for 09
• 16 Apr 2021 - slides done for 09, ready to start recording!
• 15 Apr 2021 - added slides, extra-curriculum, exercises and video notebook for 08, started making slides for 09, will finish tomorrow
• 14 Apr 2021 - recorded 12 videos for notebook 08, finished the section! time to make slides for 09 and get into it
• 10 Apr 2021 - recorded 4 videos for notebook 08
• 9 Apr 2021 - recorded 6 videos for notebook 08
• 8 Apr 2021 - recorded 10 videos for notebook 08! more coming tomorrow! home stretch baby!!!
• 7 Apr 2021 - added a whole bunch of images to notebook 08, getting ready for recording tomorrow!
• 1 Apr 2021 - added notebook 09: SkimLit, almost finished, a little cleaning and we'll be ready for slide making!
• 31 Mar 2021 - added notebook 08, going to finish tomorrow, then onto 09!
• 24 Mar 2021 - Recorded 8 videos for 07, finished! onto materials (slides/notebooks) for 08, 09
• 23 Mar 2021 - Recorded 6 videos for 07 (finally), going to finish tomorrow
• 22 Mar 2021 - Polished notebook 07 ready for recording, made slides for 07, added template for 07 (for a student to go through and practice), ready to record!
• 17 Mar 2021 - 99% finished notebook 07, added links to first 14 hours of the course on YouTube (10 hours in part 1, 4 hours in part 2)
• 11 Mar 2021 - added even more text annotations to notebook 07, finishing tomorrow, then slides
• 10 Mar 2021 - Typed a whole bunch of explanations into notebook 07, continuing tomorrow
• 09 Mar 2021 - fixed plenty of code in notebook 07, should run end to end very cleanly (though loading times are still a thing)
• 05 Mar 2021 - added draft notebook 07 (heaps of data loading and model training improvements in this one!), gonna fix up over next few days
• 01 Mar 2021 - Added slides for 06 (see them here)
• 26 Feb 2021 - 🚀 LAUNCHED!!!!! also finished recording videos for 06, onto 07, 08, 09 for next release
• 24 Feb 2021 - recorded 9 videos for section 06, launch inbound!!!
• 23 Feb 2021 - rearranged GitHub in preparation for launch 🚀
• 18 Feb 2021 - recorded 8 videos for 05 and... it's done! onto polishing the GitHub
• 17 Feb 2021 - recorded 10 videos for 05! going to finish tomorrow 🚀
• 16 Feb 2021 - polished slides for 05 and started recording videos, got 7 videos done for 05
• 15 Feb 2021 - finished videos for 04, now preparing to record for 05!
• 12 Feb 2021 - recored 7 videos for section 04... wanted 10 but we'll take 7 (🤔 this seems to have happened before)
• 11 Feb 2021 - NO PROGRESS - gave a Machine Learning deployment tutorial for Stanford's CS329s (using the model code from this course!!!) - see the full tutorial materials
• 08 Feb 2021 - recorded 10 videos for section 03... and section 03 is done! 🚀 onto section 04
• 30 Jan 2021 - 07 Feb 2021: NO PROGRESS (working on a ML deployment lecture for Stanford's CS329s... more on this later)
• 29 Jan 2021 - recorded 9 videos for section 03... closer to 10 than yesterday but still not there
• 28 Jan 2021 - recorded 7 videos for section 03... wanted 10 but we'll take 7
• 27 Jan 2021 - recorded 10 videos for section 03
• 26 Jan 2021 - polished GitHub README (what you're looking at) with a nice table
• 23 Jan 2021 - finished slides of 06
• 22 Jan 2021 - finished review of notebook 06 & started slides of 06
• 21 Jan 2021 - finished slides for 05 & started review of 06
• 20 Jan 2021 - finished notebook 05 & 95% slides for 05
• 19 Jan 2021 - found a storage idea for data during course (use Google Storage in same region as Colab Notebooks, cheapest/fastest)
• 18 Jan 2021 - reviewed notebook 05 & slides for 05
• 17 Jan 2021 - finished notebook 04 & slides for 04
• 16 Jan 2021 - review notebook 04 & made slides for transfer learning
• 13 Jan 2021 - review notebook 03 again & finished slides for 03, BIGGGGG updates to the README, notebook 03 99% done, just need to figure out optimum way to transfer data (e.g. when a student downloads it, where's best to store it in the meantime? Dropbox? S3? GS (too expensive)
• 11 Jan 2021 - reviewed notebook 03, 95% ready for recording, onto slides for 03
• 9 Jan 2021 - I'm back baby! Finished all videos for 02, now onto slides/materials for 03, 04, 05 (then I'll get back in the lab)
• 19 Dec 2020 - ON HOLD (family holiday until Jan 02 2021)
• 18 Dec 2020 - recorded 75% of videos for 02
• 17 Dec 2020 - recorded 50% of videos for 02
• 16 Dec 2020 - recorded 100% of videos for 01
• 15 Dec 2020 - recorded 90% of videos for 01
• 09 Dec 2020 - finished recording videos for 00
• 08 Dec 2020 - recorded 90% of videos for 00
• 05 Dec 2020 - trialled recording studio for ~6 videos with notebook 00 material
• 04 Dec 2020 - setup recording studio in closet
• 03 Dec 2020 - finished notebook 02, finished slides for 02, time to setup recording studio
• 02 Dec 2020 - notebook 02 95% done, slides for 02 90% done
• 01 Dec 2020 - added notebook 02 (90% polished), start preparing slides for 02
• 27 Nov 2020 - polished notebook 01, made slides for notebook 01
• 26 Nov 2020 - polished notebook 00, made slides for notebook 00

tensorflow-deep-learning's Issues

Cell 24 fails due to the same breaking change of TensorFlow 2.7

In cell 24,

# Make predictions with our trained model
y_reg_preds = model_3.predict(y_reg_test)

fails due to the same shape error raised in #278

It should be (at least in my local m1 machine)

# Make predictions with our trained model
y_reg_preds = model_3.predict(tf.expand_dims(y_reg_test, axis=-1))

Help with your assignment-build model_5 with custom embedding like model _1

From: video 285, Comparing the performance of all of our modelling experiments
Time in video: 3:53 / 9:36
Demo: https://colab.research.google.com/drive/1qrg2y7bTCVZwWS_xxpBQ8i-Y3EMu1wij?usp=sharing#scrollTo=b5147e40-b745-4e1d-9c9c-730e42ee4501

My implementation code:

token_inputs = layers.Input(shape=(1,), dtype="string", name="token_inputs")
x = text_vectorizer(token_inputs)
x = token_embed(x)
x = layers.GlobalAveragePooling1D()(x)
token_outputs = layers.Dense(128, activation="relu")(x)

When fitting the model with the code above, I am getting a nan loss.

Original (Your code):

token_inputs = layers.Input(shape=[], dtype="string", name="token_inputs")
token_embeddings = tf_hub_embedding_layer(token_inputs)
token_outputs = layers.Dense(128, activation="relu")(token_embeddings)

Python deep learning tensorflow

First learning model doesnt work.

On your first learning model at roughly 4:45:00 into the video. Google colab is having an issue with I believe the model.fit(X, y, epochs=5) .

I have adjusted the code a few times, trying to debug the system. Though am still very new at this. Any advice would help.

plotting augmented_img broken

When plotting augmented_img we obtain the same image as the original random image. This was confirmed by rerunning the original class notebook.

Data augmentation - rotation_range is an integer not a float

In the Computer Vision and Convolutional Neural Network and data augmentation sections

https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/03_convolutional_neural_networks_in_tensorflow.ipynb

we should emphasize that rotation_range is an int. Putting 0.2 equates finally to zero

Fix NBEATS first subtract block

The NBEATS architecture rebuild is slightly off (I think, as of today I haven't read the paper in almost a year).

There's a missing subtract connection (the red arrow below).

cjtwigt pointed this out on Discord (text below is his quote):

Having said all this.. I think I have found an error in video lecture 336 where you build the NBEATS model. The way you have constructed the model the 2nd NBEATS block does not receive the model inputs via the first subtract layer (the direct line from the model inputs to the first subtract layer is missing). I have constructed it this way (and this way it beats the naive results by 0.5%)):

tf.random.set_seed(42)

nbeats_block_layer = NBeatsBlock(
    input_size=INPUT_SIZE,
    theta_size=THETA_SIZE,
    horizon=HORIZON,
    n_neurons=N_NEURONS,
    n_layers=N_LAYERS,
    name="InitialBlock"
)

inputs = layers.Input(shape=(WINDOW_SIZE,), name="stack_input")

residuals, forecast = nbeats_block_layer(inputs)

x = inputs

for i, _ in enumerate(range(N_STACKS-1)):
    x = layers.subtract([x, residuals], name=f"subtract{i}")
    residuals, block_forecast = NBeatsBlock(
        input_size=INPUT_SIZE,
        theta_size=THETA_SIZE,
        horizon=HORIZON,
        n_neurons=N_NEURONS,
        n_layers=N_LAYERS,
        name=f"NBeatsBlock{i}"
    )(x)
    forecast = layers.add([forecast, block_forecast], name=f"add{i}")

outputs = forecast
               
model_7 = tf.keras.Model(inputs, outputs, name="model_7_NBEATS")

Original paper: https://arxiv.org/abs/1905.10437

See the notebook it's related to here (see model_7 creation): https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/10_time_series_forecasting_in_tensorflow.ipynb

rgba to rgb

How do you create a loop to strip down the 4th colour channel in rgba images to rgb for the whole dataset?

  _tfio.experimental.color.rgba_to_rgb(
  img, name=None
  )_

I tried using the above cell to pre-process images but it yields a transformation for only one image at a time and the resultant shape is not saved for image processing.

plot_decision_boundary for model 7 reads "doing multiclass classification"

In trying to visualize the metrics for model_7, I ran plot_boundary_decision(model_7, X, y) and it prints doing multiclass classification. I've tried adding the input_shape parameter to the first layer, as well as expand_dims in the model_7.fit function call, and these result in errors that logits and labels have to be the same dimension.

The plot drawn is all orange and doesn't look like the video, but the data in the model is the same as the video.

In notebook 5, GlobalAveragePooling2D() does not give the same accuracy and loss

Hey Daniel,

[EfficientNetB0]
I tried running the notebook with a slight change where I used ImageDataGenerator and flow_from_dataframe instead of image_dataset_from_directory for data preparation and when I train the same model(EfficientNetB0) as in notebook 5 which uses GlobalAveragePooling2D(). The loss and accuracy are far far very far less and infact stops improving as compared to the losses and accuracy you got while training.

I thought I had made mistake in my data processing but then I double checked by running the different model from notebook 4(EfficientNetB0 from tf hub) with worked fine cause I saw the loss going down and accuracy improving like it should as demonstrated by you in the notebook 4. So, the data preparation is fine.

Now, I'm wondering why loss and accuracy is note improving when using the model in notebook 5. Is there something wrong with the model? or data?

Changes made on the default parameters of Text Vectorizer

The parameter in Text Vectorizer, which is pad_to_max_tokens has been changes from True to False.

From The Video:

From the Docs

I don't know if it will make my model perform worse or not. I haven't trained my model yet. But I did notice something which is, even if I set pad_to_max_tokens= False which is by default then also it is padding the integer tensor with zeroes at the end

So I am a bit confused now. Let me train my model and check the accuracy score then I think the confusion will go away.

My Text Vectorizer Config:

PyTorch Deep Learning

PyTorch Deep Learning,

I think it would be really interesting and I like how @mrdbourke teaches.

With best regards,
Ranuga

Update ImageDataGenerator to image_dataset_from_directory

This will greatly save time for some students instead of waiting for the kernel to finish the run. As a bonus, to teach students how to pipeline the input better. Thanks to your "documentation reading" skills, I have learned to improve things because I hate to wait.

Current behaviour

Your code in 03_convolutional_neural_networks_in_tensorflow uses ImageDataGenerator. As you said, this method causes more delay in every epoch because it augments the data when it loaded the model.

train_datagen_augmented = ImageDataGenerator(rescale=1/255.,
                                             rotation_range=20, 
                                             shear_range=0.2,
                                             zoom_range=0.2, 
                                             width_shift_range=0.2, 
                                             height_shift_range=0.2, 
                                             horizontal_flip=True


train_data_augmented = train_datagen_augmented.flow_from_directory(train_dir,
                                                                   target_size=(224, 224),
                                                                   batch_size=32,
                                                                   class_mode='categorical') 

Suggestion

Since tf.keras.preprocessing is marked as deprecated (but still usable), maybe we can use tf.keras.utils.image_dataset_from_directory as to utilize tf.data.Dataset.

from tensorflow.keras import layers, models
tf.random.set_seed(42)

train_augmented = tf.keras.utils.image_dataset_from_directory(
    train_dir,
    label_mode='categorical',
    image_size=(256, 256),
    batch_size=32,
    seed=42
)

preprocessing_layer  = models.Sequential([
    layers.Rescaling(1./255),
    layers.RandomFlip(mode="horizontal"),
    layers.RandomRotation((0.0, 0.01)),
    layers.RandomZoom((0.0, 0.1)),
    layers.RandomTranslation((0.0, 0.1), (0.0, 0.1))

])

train_augmented = (
    train_augmented
    .map(
        lambda x, y: (preprocessing_layer(x), y),
        num_parallel_calls=tf.data.AUTOTUNE
        )
    .cache()
    .prefetch(buffer_size=tf.data.AUTOTUNE)

)

Benchmark

Tested on multiclass classification with 20 classes without sampling. 15000 training files and 5000 validation files. Although this dataset is not similar to the notebook example, it shows how it can benefit smaller datasets as well.

ImageDataGenerator takes 11 minutes and 19.4 seconds.
image_dataset_from_directory takes 1 minute and 37.5 seconds.
Improvement speed 6.97 times over normal run.

System information

• OS Platform and Distribution: Windows 10
• TensorFlow version: v2.8.0
• GPU model and memory: RTX 3060 Ti 8GB

07_food_vision_milestone_project_1.ipynb - In:29

So when I run cell 29 I get the following error. The GPU is "GPU 0: Tesla T4 (UUID: GPU-6b3d416d-69cc-a08d-bde3-f0e6c29fdd87)".

---

TypeError Traceback (most recent call last)

in
3 # Create base model
4 input_shape = (224, 224, 3)
----> 5 base_model = tf.keras.applications.EfficientNetB0(include_top=False)
6 base_model.trainable = False # freeze base model layers
7

4 frames

/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.traceback)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb

TypeError: Exception encountered when calling layer "tf.math.truediv" (type TFOpLambda).

x and y must have the same dtype, got tf.float16 != tf.float32.

Call arguments received by layer "tf.math.truediv" (type TFOpLambda):
• x=tf.Tensor(shape=(None, None, None, 3), dtype=float16)
• y=tf.Tensor(shape=(3,), dtype=float32)
• name=None

Loading Checkpoint performance assertion fails

I tried to save model to a checkpoint according to this notebook, but the assertion fails. I check the github saved notebook here https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/07_food_vision_milestone_project_1.ipynb
The same assertion happens. Any ideas why this is happening?

02_neural_network_classification_with_tensorflow - wrong shape in model_3

We are doing binary classification, expecting the predictions to be 0 or 1. For model_2, the shape of model_2.predict(X) is (1000, 1), i.e., for each input row we get a scalar result. However, the first layer in the TF 2.7.0+ tweak to model_3,

tf.keras.layers.Dense(100, input_shape=(None, 1))

seems to be wrong: the shape of model_3.predict(X) is (1000, 2, 1), i.e., for each input row we now get 2 numbers. This is also indicated by the function plot_decision_boundary(), saying 'doing multiclass classification...' and showing different contours than those shown in the video.

Based on the documentation of Dense and Input, I believe that the input_shape should be (2,), i.e., using

tf.keras.layers.Dense(100, input_shape=(2,))

or we can just keep the model as it was originally, with

tf.keras.layers.Dense(100)

This throws no errors and produces the same plot as shown in the video, with 'doing binary classification'.

Compare the original plot to the plot corresponding to the tweaked version:

None of the .ipynb files is opening

Hi!
I have a problem loading the .ipynb files from every notebook that you have. In every notebook I load it gives back the same error message "Something went wrong"

02_neural_network_classification_in_tensorflow - plot_decision_boundary function issue

while building plot_decision_boundary it is giving error as xx and yy not define.

Even though below is already define.

x_min,x_max = X[:, 0].min() - 0.1,X[:, 0].max() + 0.1
y_min,y_max = X[:, 1].min() - 0.1,X[:, 1].max() + 0.1
xx, yy = np.meshgrid(np.linspace(x_min,x_max,100),
np.linspace(y_min,y_max,100))
Please advise

Huber in Regression Evaluation Metrics

Hi, Daniel. Thanks for your great video!
About the Huber in Regression Evauation Metrics:

I found it confusing when you describe Huber in "when to use" as a combination of MAE and MSE because the first impression of mine is let's use Huber to balance the advantage and drawbacks between MAE and MSE since it's a combination.
However, I found the chart here to show that Huber would be much more insensitive anytime if we compare to MAE and MSE no matter what degree it would be.

The link is here: https://www.researchgate.net/figure/Comparison-of-the-MSE-MAE-and-Huber-losses_fig3_340644261
Yes, Huber does show a milder reaction to a larger value by the square. But it seems Huber is not that a "combination".
Perhaps we may say it's an "improvement" or an "alternative" would be not that confusing?
Thanks, Daniel!. You're the best. Learning a lot from your videos. Thanks! - Alvin

Error in calculating most_wrong in 08_introduction_to_nlp_in_tensorflow.ipynb (its in video notebook also)

CURRENT CODE

#Create dataframe with validation sentences and best performing model predictions
val_df = pd.DataFrame({"text": val_sentences,
                       "target": val_labels,
                       "pred": model_6_pretrained_preds,
                       "pred_prob": tf.squeeze(model_6_pretrained_pred_probs)})
val_df.head()

# Find the wrong predictions and sort by prediction probabilities
most_wrong = val_df[val_df["target"] != val_df["pred"]].sort_values("pred_prob", ascending=False)
most_wrong[:10]

SUGGESTED NEW CODE

# Create dataframe with validation sentences and best performing model predictions, and prediction errors
val_df = pd.DataFrame({"text": val_sentences,
                       "target": val_labels,
                       "pred": model_6_pretrained_preds,
                       "pred_prob":tf.squeeze(model_6_pretrained_pred_probs), 
                       "pred_error":tf.abs(tf.squeeze(model_6_pretrained_pred_probs)-val_labels)})     #### FIX (need comma on prev line too)
val_df.head()   

# Find the wrong predictions and sort by prediction error
most_wrong = val_df[val_df["target"] != val_df["pred"]].sort_values("pred_error", ascending=False)
most_wrong.head(30)      #### FIX look at 30 so we see some cases on wrong 1s and wrong 0s
# Check from the bottom as well - we should see least wrong cases  - predictions just a little on the wrong side of 0.50
most_wrong[-10:]

The Spacy Code in Skimlit Excersie is giving an error

The Code giving Error:

from spacy.lang.en import English
nlp = English() 
sentencizer = nlp.create_pipe("sentencizer") 
nlp.add_pipe(sentencizer) 
doc = nlp(example_abstracts[0]["abstract"]) 
abstract_lines = [str(sent) for sent in list(doc.sents)]
abstract_lines

This code is from the notebook link: https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/09_SkimLit_nlp_milestone_project_2.ipynb

When Trying to make predictions on Custom RCT data.

The Error

ValueError                                Traceback (most recent call last)

[<ipython-input-285-931098a41489>](https://localhost:8080/#) in <module>
      3 nlp = English() # setup English sentence parser
      4 sentencizer = nlp.create_pipe("sentencizer") # create sentence splitting pipeline object
----> 5 nlp.add_pipe(sentencizer) # add sentence splitting pipeline object to sentence parser
      6 doc = nlp(example_abstracts[0]["abstract"]) # create "doc" of parsed sequences, change index for a different abstract
      7 abstract_lines = [str(sent) for sent in list(doc.sents)] # return detected sentences from doc in string type (not spaCy token type)

[/usr/local/lib/python3.7/dist-packages/spacy/language.py](https://localhost:8080/#) in add_pipe(self, factory_name, name, before, after, first, last, source, config, raw_config, validate)
    771             bad_val = repr(factory_name)
    772             err = Errors.E966.format(component=bad_val, name=name)
--> 773             raise ValueError(err)
    774         name = name if name is not None else factory_name
    775         if name in self.component_names:

ValueError: [E966] `nlp.add_pipe` now takes the string name of the registered component factory, not a callable component. Expected string, but got <spacy.pipeline.sentencizer.Sentencizer object at 0x7fe326b3ba00> (name: 'None').

- If you created your component with `nlp.create_pipe('name')`: remove nlp.create_pipe and call `nlp.add_pipe('name')` instead.

- If you passed in a component like `TextCategorizer()`: call `nlp.add_pipe` with the string name instead, e.g. `nlp.add_pipe('textcat')`.

- If you're using a custom component: Add the decorator `@Language.component` (for function components) or `@Language.factory` (for class components / factories) to your custom component and assign it a name, e.g. `@Language.component('your_name')`. You can then run `nlp.add_pipe('your_name')` to add it to the pipeline.

The code I wrote which works for predicting on custom RCT data:

from spacy.lang.en import English
nlp= English()
nlp.add_pipe("sentencizer")
doc= nlp(example_abstracts[0]["abstract"])
abstract_lines= [str(sent) for sent in list(doc.sents)]
abstract_lines

Online book version of course formatting issue

The following two pages are showing up weird:

1. https://dev.mrdbourke.com/tensorflow-deep-learning/08_introduction_to_nlp_in_tensorflow/
2. https://dev.mrdbourke.com/tensorflow-deep-learning/09_SkimLit_nlp_milestone_project_2/

The menu on the right interferes with the text of the book.

No idea why.

Notebook 03: `tf.keras.preprocessing.image.ImageDataGenerator` is deprecated

From Pierre in the Discord:

Hey Daniel in the TF Developer Certificate course you cover the ImageDataGenerator quite a bit, also for augmenting data. On udemy a few people already pointed out that this is depreciated. It still works as of now but the tf documentary points towards using preprocessing layers instead or tf.keras.utils.image_dataset_from_directory . Maybe it would be good to point that out on a little note before the videos, to avoid confusion. Also the process is a little different. Otherwise great course as usual https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator

Code typo in notebook tensorflow-deep-learning/01_neural_network_regression_in_tensorflow.ipynb

Under the section titled "Comparing results", the code below:
model_results = [["model_1", mae_1, mse_1], ["model_2", mae_2, mse_2], ["model_3", mae_3, mae_3]]
should be changed to
model_results = [["model_1", mae_1, mse_1], ["model_2", mae_2, mse_2], ["model_3", mae_3, mse_3]]
This change would properly reflect the mse for the third model instead of showing the mae twice.

Speed of fitting model is hours for some reason, 07_food_vision_milestone_project. Also Found a resolution for the mixed precision float16 - float32 error

  Also running into a problem. It appears that the ETA for fitting the model after downgrading to 2.4.1 is running at like 100 hours even with using the Nvida T4. Even running the project directly from github https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/07_food_vision_milestone_project_1.ipynb it still is getting 100+ hours

Saving TensorBoard log files to: training_logs/efficientnetb0_101_classes_all_data_feature_extract/20221227-085937
Epoch 1/3
3/2368 [..............................] - ETA: 97:10:20 - loss: nan - accuracy: 0.0382

Tue Dec 27 08:59:33 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 66C P0 31W / 70W | 3MiB / 15109MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

Turn on mixed precision training

from tensorflow.keras import mixed_precision
mixed_precision.set_global_policy(policy="mixed_float16") # set global policy to mixed precision

WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING
The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once

Originally posted by @Jayms8462 in #82 (comment)

Content 212 fixed

The issue has been fixed in TF 2.10

#time7:50:43- i got error

TypeError: unsupported operand type(s) for +=: 'Dense' and 'str'
daniel I followed ur steps and I tried to create a model2. I am facing an issue for more than two days can u help me to figure out this problem. I will attach the full error.
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:855 train_function *
return step_function(self, iterator)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:845 step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:1285 run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:2833 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:3608 _call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:838 run_step **
outputs = model.train_step(data)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:795 train_step
y_pred = self(x, training=True)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py:1018 call
name_scope = self._name_scope() # Avoid autoincrementing.
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py:2387 _name_scope
name_scope += '/'

TypeError: unsupported operand type(s) for +=: 'Dense' and 'str'

Incorrect behaviour of plot_decision_boundary() in 02_neural_network_classification_with_tensorflow.ipynb

Hello,

I am working through video 76 on the "TensorFlow Developer Certificate in 2022: Zero to Mastery" course. The function plot_decision_boundary() has the following logic:

if len(y_pred[0]) > 1:
    print("doing multiclass classification...")
    # We have to reshape our predictions to get them ready for plotting
    y_pred = np.argmax(y_pred, axis=1).reshape(xx.shape)
  else:
    print("doing binary classifcation...")
    y_pred = np.round(y_pred).reshape(xx.shape) #rounding to give us a 0 or 1

When I check len(y_pred[0]) which comes from y_pred = model.predict(x_in), I get a length of 2. So the current function then follows the logic for multi-class classification, although the problem being worked on is binary. Can this be fixed?

Cheers!
Donal

plot_and_pred function

In the explanation for binary image classification https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/03_convolutional_neural_networks_in_tensorflow.ipynb you defined class_names which were later used in pred_and_plot function to round the probability >0.5 to 1 and <0.5 to 0 to refer to steak and pizza, respectively. But unfortunately the references are 1 for steak and 2 for pizza due to ".DS_store" file at index 0.

If you look at the prediction (pizza_dad.jpeg) right before the start of the section on multi class classification, it predicts ".DS_store" where it should've been pizza, and the prediction above that is given pizza where it should be steak - The model is predicting right but comes out wrong due to class_names reference. Perhaps class_names = class_names[1:] will help.

Error in Time Series model_10

mae value did not match when evaluated using mode_10.evaluate() and using evaluate_time_series()
as tf.squeeze was not performed on the predictions(turkey_preds).
we have used mae as the loss function, so evaluate() returned 638.3 which is the correct value not 17144.766

Placeholder issue for BitPredict project

From alex.lit in the Discord:

Will remove this issue if alex.lit posts here

M1 Usage Notes

I'm currently running everything in tensorflow metal out of my M1 laptop, there are a couple of notes I would like to raise if someone else is doing the same.

Some of the functions are not added to tensorflow metal, leading to a no device error like the following

NotFoundError: No registered 'RngReadAndSkip' OpKernel for 'GPU' devices compatible with node {{node RngReadAndSkip}}
	.  Registered:  device='CPU'
 [Op:RngReadAndSkip]

As this forum mentioned https://developer.apple.com/forums/thread/695319, RngReadAndSkip is yet to be added to the tensorflow metal plugin, the temporary workaround is,

with tf.device('/cpu:0'):
    #Your code here

This would happen on all data argumentation steps "Random Flip"

Tenserflow is very helpful

Question about model 4 conv1d in notebook 10(time series prediction for bitcoin price)

This is the model summary in model 4

Model: "model_4_conv1D"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lambda_1 (Lambda)            (None, 1, 7)              0         
_________________________________________________________________
conv1d (Conv1D)              (None, 1, 128)            4608      
_________________________________________________________________
dense_6 (Dense)              (None, 1, 1)              129       
=================================================================
Total params: 4,737
Trainable params: 4,737
Non-trainable params: 0

Don't we want input to conv1d layer have shape of (batch_size, timesteps(window size), input_dim) ?

Transfer Learning with TensorFlow Part 2: Fine-tuning Data Augmentation

i have same code this notebook but my image is not augmented same as before

Notebook 07: Cannot save model checkpoint for FoodVision Big

Getting an error when training FoodVision Big:

# Fit the model with callbacks
history_101_food_classes_feature_extract = model.fit(train_data, 
                                                     epochs=3,
                                                     steps_per_epoch=len(train_data),
                                                     validation_data=test_data,
                                                     validation_steps=int(0.15 * len(test_data)),
                                                     callbacks=[create_tensorboard_callback("training_logs", 
                                                                                            "efficientnetb0_101_classes_all_data_feature_extract"),
                                                                model_checkpoint])

>>>WARNING:tensorflow:Can save best model only with val_acc available, skipping.

Looks like it's an issue with the model_checkpoint callback.

This causes the assertion for the cloned model later on to fail:

# Evalaute cloned model with loaded weights (should be same score as trained model)
results_cloned_model_with_loaded_weights = cloned_model.evaluate(test_data)

>>> ---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
/tmp/ipykernel_1443486/1110829135.py in <module>
      1 # Loaded checkpoint weights should return very similar results to checkpoint weights prior to saving
      2 import numpy as np
----> 3 assert np.isclose(results_feature_extract_model, results_cloned_model_with_loaded_weights).all() # check if all elements in array are close

AssertionError: 

Need to update the model checkpoint to make sure it can save a model whilst training.

I was unable to implement BERT trained on PubMed

All steps were carried out according to the documentation, and when attempted to train the model after compiling, I received this error:

Epoch 1/20
---------------------------------------------------------------------------
UnknownError                              Traceback (most recent call last)
[<ipython-input-98-f1f15d667bfd>](https://localhost:8080/#) in <module>
      5                               epochs=20,
      6                               callbacks=[model_checkpoint,
----> 7                                          early_stopping])

1 frames
[/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py](https://localhost:8080/#) in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     53     ctx.ensure_initialized()
     54     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 55                                         inputs, attrs, num_outputs)
     56   except core._NotOkStatusException as e:
     57     if name is not None:

UnknownError: Graph execution error:

Detected at node 'cond/CudnnRNNV3' defined at (most recent call last):
    File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
      "__main__", mod_spec)
    File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
      exec(code, run_globals)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py", line 16, in <module>
      app.launch_new_instance()
    File "/usr/local/lib/python3.7/dist-packages/traitlets/config/application.py", line 846, in launch_instance
      app.start()
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelapp.py", line 612, in start
      self.io_loop.start()
    File "/usr/local/lib/python3.7/dist-packages/tornado/platform/asyncio.py", line 132, in start
      self.asyncio_loop.run_forever()
    File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
      self._run_once()
    File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
      handle._run()
    File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
      self._context.run(self._callback, *self._args)
    File "/usr/local/lib/python3.7/dist-packages/tornado/ioloop.py", line 758, in _run_callback
      ret = callback()
    File "/usr/local/lib/python3.7/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1233, in inner
      self.run()
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1147, in run
      yielded = self.gen.send(value)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 365, in process_one
      yield gen.maybe_future(dispatch(*args))
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper
      yielded = next(result)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 268, in dispatch_shell
      yield gen.maybe_future(handler(stream, idents, msg))
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper
      yielded = next(result)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 545, in execute_request
      user_expressions, allow_stdin,
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper
      yielded = next(result)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/ipkernel.py", line 306, in do_execute
      res = shell.run_cell(code, store_history=store_history, silent=silent)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/zmqshell.py", line 536, in run_cell
      return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2855, in run_cell
      raw_cell, store_history, silent, shell_futures)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2881, in _run_cell
      return runner(coro)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner
      coro.send(None)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3058, in run_cell_async
      interactivity=interactivity, compiler=compiler, result=result)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3249, in run_ast_nodes
      if (await self.run_code(code, result,  async_=asy)):
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3326, in run_code
      exec(code_obj, self.user_global_ns, self.user_ns)
    File "<ipython-input-98-f1f15d667bfd>", line 7, in <module>
      early_stopping])
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1409, in fit
      tmp_logs = self.train_function(iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1051, in train_function
      return step_function(self, iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1040, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1030, in run_step
      outputs = model.train_step(data)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 889, in train_step
      y_pred = self(x, training=True)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 490, in __call__
      return super().__call__(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 459, in call
      inputs, training=training, mask=mask)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph
      outputs = node.layer(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/bidirectional.py", line 249, in __call__
      return super(Bidirectional, self).__call__(inputs, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/bidirectional.py", line 365, in call
      initial_state=forward_state, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/base_rnn.py", line 515, in __call__
      return super(RNN, self).__call__(inputs, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/lstm.py", line 673, in call
      runtime) = lstm_with_backend_selection(**normal_lstm_kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/lstm.py", line 1184, in lstm_with_backend_selection
      gru_lstm_utils.function_register(defun_gpu_lstm, **params)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/gru_lstm_utils.py", line 244, in function_register
      concrete_func = func.get_concrete_function(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/lstm.py", line 1152, in gpu_lstm_with_fallback
      false_fn=stardard_lstm_fn)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/lstm.py", line 1132, in cudnn_lstm_fn
      return_sequences=return_sequences)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/rnn/lstm.py", line 1005, in gpu_lstm
      time_major=time_major)
Node: 'cond/CudnnRNNV3'
Fail to find the dnn implementation.
	 [[{{node cond/CudnnRNNV3}}]]
	 [[tribrid_embedding_model/bidirectional_3/forward_lstm_3/PartitionedCall]] [Op:__inference_train_function_227513]

Something else I noted, after installing and importing tensorflow_text, I was required to restart the runtime, and then models with bidirectional LSTM layers would not be able to train. In this next case, the traceback error is:

Saving TensorBoard log files to: SkimLit-logs/model_1_tok_emb_conv1d/20220902-121155
Epoch 1/3
---------------------------------------------------------------------------
UnimplementedError                        Traceback (most recent call last)
[<ipython-input-48-9b6dc8cf5719>](https://localhost:8080/#) in <module>
      6                               validation_steps=int(0.1*len(valid_dataset)),
      7                               callbacks=[create_tensorboard_callback(SAVE_DIR,
----> 8                                                                      "model_1_tok_emb_conv1d")])

1 frames
[/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py](https://localhost:8080/#) in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     53     ctx.ensure_initialized()
     54     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 55                                         inputs, attrs, num_outputs)
     56   except core._NotOkStatusException as e:
     57     if name is not None:

UnimplementedError: Graph execution error:

Detected at node 'model_1_dense/conv_1D_layer/Conv1D' defined at (most recent call last):
    File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
      "__main__", mod_spec)
    File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
      exec(code, run_globals)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py", line 16, in <module>
      app.launch_new_instance()
    File "/usr/local/lib/python3.7/dist-packages/traitlets/config/application.py", line 846, in launch_instance
      app.start()
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelapp.py", line 612, in start
      self.io_loop.start()
    File "/usr/local/lib/python3.7/dist-packages/tornado/platform/asyncio.py", line 132, in start
      self.asyncio_loop.run_forever()
    File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
      self._run_once()
    File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
      handle._run()
    File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
      self._context.run(self._callback, *self._args)
    File "/usr/local/lib/python3.7/dist-packages/tornado/ioloop.py", line 758, in _run_callback
      ret = callback()
    File "/usr/local/lib/python3.7/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1233, in inner
      self.run()
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1147, in run
      yielded = self.gen.send(value)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 381, in dispatch_queue
      yield self.process_one()
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 346, in wrapper
      runner = Runner(result, future, yielded)
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1080, in __init__
      self.run()
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1147, in run
      yielded = self.gen.send(value)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 365, in process_one
      yield gen.maybe_future(dispatch(*args))
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper
      yielded = next(result)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 268, in dispatch_shell
      yield gen.maybe_future(handler(stream, idents, msg))
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper
      yielded = next(result)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 545, in execute_request
      user_expressions, allow_stdin,
    File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper
      yielded = next(result)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/ipkernel.py", line 306, in do_execute
      res = shell.run_cell(code, store_history=store_history, silent=silent)
    File "/usr/local/lib/python3.7/dist-packages/ipykernel/zmqshell.py", line 536, in run_cell
      return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2855, in run_cell
      raw_cell, store_history, silent, shell_futures)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2881, in _run_cell
      return runner(coro)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner
      coro.send(None)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3058, in run_cell_async
      interactivity=interactivity, compiler=compiler, result=result)
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3249, in run_ast_nodes
      if (await self.run_code(code, result,  async_=asy)):
    File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3326, in run_code
      exec(code_obj, self.user_global_ns, self.user_ns)
    File "<ipython-input-48-9b6dc8cf5719>", line 8, in <module>
      "model_1_tok_emb_conv1d")])
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1409, in fit
      tmp_logs = self.train_function(iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1051, in train_function
      return step_function(self, iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1040, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1030, in run_step
      outputs = model.train_step(data)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 889, in train_step
      y_pred = self(x, training=True)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 490, in __call__
      return super().__call__(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 459, in call
      inputs, training=training, mask=mask)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph
      outputs = node.layer(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/convolutional/base_conv.py", line 250, in call
      outputs = self.convolution_op(inputs, self.kernel)
    File "/usr/local/lib/python3.7/dist-packages/keras/layers/convolutional/base_conv.py", line 232, in convolution_op
      name=self.__class__.__name__)
Node: 'model_1_dense/conv_1D_layer/Conv1D'
DNN library is not found.
	 [[{{node model_1_dense/conv_1D_layer/Conv1D}}]] [Op:__inference_train_function_12429]

Anyone having these issues within Colab? Thanks

You can see the error accessing my colab, but it has lots there. It will take some time to run and reach to the latest part.
https://colab.research.google.com/drive/1wRbbEeLDmD4gNjlWvCmxYtDxc8lliaNw?usp=sharing

mixed precision with TPU

Hello,

On colab, I don't have the GPU with minimum required compute capability for mixed precision. So I decided to use TPU and set the policy as 'mixed_bfloat16'. However, while fitting the feature extraction model, I get the following error. I am not able to find solutions online as well. I am also not sure if this is because I am using TPU or if there is something I am missing. Any help would be great.This is the error:

/usr/local/lib/python3.7/dist-packages/keras/utils/generic_utils.py:497: CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
category=CustomMaskWarning)
Epoch 1/2

InvalidArgumentError Traceback (most recent call last)
in ()
----> 1 hist1 = model.fit(train_data,steps_per_epoch=len(train_data),epochs=2, validation_data=test_data,validation_steps=int(0.15*len(test_data)),callbacks = [callback1,callback2])

6 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:

InvalidArgumentError: No OpKernel was registered to support Op 'DepthwiseConv2dNativeBackpropInput' used by {{node gradient_tape/model_7/efficientnetb0/block7a_dwconv/depthwise/DepthwiseConv2dNativeBackpropInput}} with these attrs: [dilations=[1, 1, 1, 1], T=DT_BFLOAT16, data_format="NHWC", strides=[1, 1, 1, 1], explicit_paddings=[], padding="SAME"]
Registered devices: [CPU]
Registered kernels:
device='XLA_CPU_JIT'; T in [DT_FLOAT, DT_DOUBLE, DT_BFLOAT16, DT_HALF]
device='XLA_GPU_JIT'; T in [DT_FLOAT, DT_DOUBLE, DT_BFLOAT16, DT_HALF]
device='GPU'; T in [DT_DOUBLE]
device='GPU'; T in [DT_FLOAT]
device='GPU'; T in [DT_HALF]
device='CPU'; T in [DT_DOUBLE]
device='CPU'; T in [DT_FLOAT]
device='CPU'; T in [DT_HALF]
device='GPU'; label='cudnn_grouped_convolution'; T in [DT_DOUBLE]
device='GPU'; label='cudnn_grouped_convolution'; T in [DT_FLOAT]
device='GPU'; label='cudnn_grouped_convolution'; T in [DT_HALF]

 [[gradient_tape/model_7/efficientnetb0/block7a_dwconv/depthwise/DepthwiseConv2dNativeBackpropInput]] [Op:__inference_train_function_197064]

I get the same error with different versions of EfficientNet.

InvalidArgumentError: Graph execution error: Video 136, Model_10

My Code Of The Model After Data Augmentation. Model cloning was not working on my TensorFlow version for unknown reasons so I used this

model_10 = Sequential([
 Conv2D(10, 3, input_shape=(224, 224, 3)),
 Activation(activation='relu'),
 Conv2D(10, 3, activation='relu'),
 MaxPool2D(),
 Conv2D(10, 3, activation='relu'),
 Conv2D(10, 3, activation='relu'),
 MaxPool2D(),
 Flatten(),
 Dense(10, activation="softmax")                     
])

model_10.compile(loss='categorical_crossentropy',
                 optimizer=tf.keras.optimizers.Adam(),
                 metrics=['accuracy'])

history_10 = model_10.fit(train_data_augmented, 
                          epochs=5, 
                          steps_per_epoch=len(train_data_augmented),
                          validation_data=test_data,
                          validation_steps=len(test_data))

Error Message

input to reshape is a tensor with 9412800 values, but the requested shape requires a multiple of 28090
[[{{node sequential_9/flatten_9/Reshape}}]] [Op:__inference_train_function_20412]

Issue with Tensorflow: Import "tensorflow.keras" could not be resolved

Hi, I am trying to use image augmentation and getting this issue. Although codes are running, data is not being augmented. Do you think this is some bug in Tensorflow?

Here is the code from Video 165

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.layers.experimental import preprocessing

# Create a data augmentation stage with horizontal flipping, rotations, zooms
data_augmentation = keras.Sequential([
  preprocessing.RandomFlip("horizontal"),
  preprocessing.RandomRotation(0.2),
  preprocessing.RandomZoom(0.2),
  preprocessing.RandomHeight(0.2),
  preprocessing.RandomWidth(0.2),
], name ="data_augmentation")

import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import os
import random
target_class = random.choice(train_data_1_percent.class_names) # choose a random class
target_dir = "10_food_classes_1_percent/train/" + target_class # create the target directory
random_image = random.choice(os.listdir(target_dir)) # choose a random image from target directory
random_image_path = target_dir + "/" + random_image # create the choosen random image path
img = mpimg.imread(random_image_path) # read in the chosen target image
plt.imshow(img) # plot the target image
plt.title(f"Original random image from class: {target_class}")
plt.axis(False); # turn off the axes

# Augment the image
augmented_img = data_augmentation(tf.expand_dims(img, axis=0)) # data augmentation model requires shape (None, height, width, 3)
plt.figure()
plt.imshow(tf.squeeze(augmented_img)/255.) # requires normalization after augmentation
plt.title(f"Augmented random image from class: {target_class}")
plt.axis(False);

The output of the plt.imshow() is same for both img and augmented_img.

Error in evaluation metrics in time series notebook for Turkey model

When evaluating the turkey model it looks like there is a shape problem causing the MAE and other metric results from evaluate_preds() to appear higher than it should be at the end of the time series notebook. I believe that tf.squeeze() should be used on y_test

turkey_results = evaluate_preds(y_true=tf.squeeze(y_test), y_pred = turkey_preds)

Does Keras Automatically Use the GPU if its available.

Does Keras Automatically Use the GPU if it's available?

Best regards,

Food Vision not training

I was working on the Food Vision project and completed it. It ran completely fine yesterday, when I tried to run it today it is giving me this error.
Detected at node 'model/efficientnetv2-b0/stem_conv/Conv2D' defined at (most recent call last): File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py", line 16, in <module> app.launch_new_instance() File "/usr/local/lib/python3.7/dist-packages/traitlets/config/application.py", line 846, in launch_instance app.start() File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelapp.py", line 612, in start self.io_loop.start() File "/usr/local/lib/python3.7/dist-packages/tornado/platform/asyncio.py", line 132, in start self.asyncio_loop.run_forever() File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever self._run_once() File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once handle._run() File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.7/dist-packages/tornado/ioloop.py", line 758, in _run_callback ret = callback() File "/usr/local/lib/python3.7/dist-packages/tornado/stack_context.py", line 300, in null_wrapper return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1233, in inner self.run() File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1147, in run yielded = self.gen.send(value) File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 381, in dispatch_queue yield self.process_one() File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 346, in wrapper runner = Runner(result, future, yielded) File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1080, in __init__ self.run() File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 1147, in run yielded = self.gen.send(value) File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 365, in process_one yield gen.maybe_future(dispatch(*args)) File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper yielded = next(result) File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 268, in dispatch_shell yield gen.maybe_future(handler(stream, idents, msg)) File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper yielded = next(result) File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 545, in execute_request user_expressions, allow_stdin, File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 326, in wrapper yielded = next(result) File "/usr/local/lib/python3.7/dist-packages/ipykernel/ipkernel.py", line 306, in do_execute res = shell.run_cell(code, store_history=store_history, silent=silent) File "/usr/local/lib/python3.7/dist-packages/ipykernel/zmqshell.py", line 536, in run_cell return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2855, in run_cell raw_cell, store_history, silent, shell_futures) File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2881, in _run_cell return runner(coro) File "/usr/local/lib/python3.7/dist-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner coro.send(None) File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3058, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3249, in run_ast_nodes if (await self.run_code(code, result, async_=asy)): File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 3326, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "<ipython-input-30-4999ceb7efa4>", line 1, in <module> history_101_food_classes_feature_extract = model.fit(train_data, epochs=10,steps_per_epoch=len(train_data), validation_data=test_data, validation_steps=int(0.15 * len(test_data)),callbacks=[tensorboard_callback, checkpoint_callback, lr_scheduler_callback, learning_rate_reduce_callback]) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1409, in fit tmp_logs = self.train_function(iterator) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1051, in train_function return step_function(self, iterator) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1040, in step_function outputs = model.distribute_strategy.run(run_step, args=(data,)) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1030, in run_step outputs = model.train_step(data) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 889, in train_step y_pred = self(x, training=True) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 490, in __call__ return super().__call__(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__ outputs = call_fn(inputs, *args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 459, in call inputs, training=training, mask=mask) File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph outputs = node.layer(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 490, in __call__ return super().__call__(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__ outputs = call_fn(inputs, *args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 459, in call inputs, training=training, mask=mask) File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph outputs = node.layer(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1014, in __call__ outputs = call_fn(inputs, *args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler return fn(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/layers/convolutional/base_conv.py", line 250, in call outputs = self.convolution_op(inputs, self.kernel) File "/usr/local/lib/python3.7/dist-packages/keras/layers/convolutional/base_conv.py", line 232, in convolution_op name=self.__class__.__name__) Node: 'model/efficientnetv2-b0/stem_conv/Conv2D' DNN library is not found. [[{{node model/efficientnetv2-b0/stem_conv/Conv2D}}]] [Op:__inference_train_function_18083]

Edit:

This looks like some tensorflow, cuDnn error. Link.

Edit-2:

This solved my issue
!apt install --allow-change-held-packages libcudnn8=8.1.0.77-1+cuda11.2 Just run this command on a cell of the notebook and the error is fixed.

Couldn't find the code for Text Generation ( Part of the Tensorflow Developer Certification)

The code on Github only provides the modelling around Text Classification but not Text generation. Could you point me towards the Text Generation modules..

the link to your full courses does not work

the link to your full courses in your README.md does not work when clicked via the Github ui.

It points to https://dbourke.link/ZTMTFcourse

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).

In NBeats Algorithm:

Failed to find data adapter that can handle input: <class 'int'>, <class 'numpy.ndarray'>

Facing this issue with the latest version of Tensorflow. Classification Problem: Video: 96

model_10 = Sequential([
 Conv2D(10, 3, input_shape=(224, 224, 3)),
 Activation(activation='relu'),
 Conv2D(10, 3, activation='relu'),
 MaxPool2D(),
 Conv2D(10, 3, activation='relu'),
 Conv2D(10, 3, activation='relu'),
 MaxPool2D(),
 Flatten(),
 Dense(10, activation="softmax")                     
])

model_10.compile(loss='categorical_crossentropy',
                 optimizer=tf.keras.optimizers.Adam(),
                 metrics=['accuracy'])

history_10 = model_10.fit(train_data_augmented, 
                          epochs=5, 
                          steps_per_epoch=len(train_data_augmented),
                          validation_data=test_data,
                          validation_steps=len(test_data))

Error

Epoch 1/10
1857/1875 [============================>.] - ETA: 0s - loss: 1.0380 - accuracy: 0.6465
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-17-652a679a1f23>](https://localhost:8080/#) in <module>()
     13 
     14 norm_history = model_13.fit(tf.expand_dims(train_data_norm, axis=-1), train_labels, epochs=10,
---> 15                             validation_data=(test_data_norm, test_labels))

1 frames
[/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py](https://localhost:8080/#) in select_data_adapter(x, y)
    985         "Failed to find data adapter that can handle "
    986         "input: {}, {}".format(
--> 987             _type_name(x), _type_name(y)))
    988   elif len(adapter_cls) > 1:
    989     raise RuntimeError(

**ValueError:** **_Failed to find data adapter that can handle input: <class 'int'>, <class 'numpy.ndarray'>_**

.

Error in number of epochs in notebook tensorflow-deep-learning/01_neural_network_regression_in_tensorflow.ipynb

Under the markdown that has the text "Let's train the same model as above for a little longer. We can do this but calling fit on it again."

The number of epochs should be 200 not 100

# Try training for a little longer (100 more epochs) history_2 = insurance_model_2.fit(X_train, y_train, epochs=100, verbose=0)

should be changed to this:

# Try training for a little longer (100 more epochs) history_2 = insurance_model_2.fit(X_train, y_train, epochs=200, verbose=0)

The downloaded model from the provided link causes an assertion error while loading