rasbt/deeplearning-models

A collection of various deep learning architectures, models, and tips

Deep Learning Models

A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.

Traditional Machine Learning

Title	Dataset	Description	Notebooks
Perceptron	2D toy data	TBD
Logistic Regression	2D toy data	TBD
Softmax Regression (Multinomial Logistic Regression)	MNIST	TBD
Softmax Regression with MLxtend's plot_decision_regions on Iris	Iris	TBD

Multilayer Perceptrons

Title	Dataset	Description	Notebooks
Multilayer Perceptron	MNIST	TBD
Multilayer Perceptron with Dropout	MNIST	TBD
Multilayer Perceptron with Batch Normalization	MNIST	TBD
Multilayer Perceptron with Backpropagation from Scratch	MNIST	TBD

Convolutional Neural Networks

Basic

Title	Dataset	Description	Notebooks
Convolutional Neural Network	TBD	TBD
CNN with He Initialization	TBD	TBD

Concepts

Title	Dataset	Description	Notebooks
Replacing Fully-Connected by Equivalent Convolutional Layers	TBD	TBD

---

AlexNet

Title	Dataset	Description	Notebooks
AlexNet Trained on CIFAR-10	TBD	TBD
AlexNet with Grouped Convolutions Trained on CIFAR-10	TBD	TBD

DenseNet

Title	Description	Daset	Notebooks
DenseNet-121 Digit Classifier Trained on MNIST	TBD	TBD
DenseNet-121 Image Classifier Trained on CIFAR-10	TBD	TBD

Fully Convolutional

Title	Dataset	Description	Notebooks
"All Convolutionl Net" -- A Fully Convolutional Neural Network	TBD	TBD

LeNet

Title	Dataset	Description	Notebooks
LeNet-5 on MNIST	TBD	TBD
LeNet-5 on CIFAR-10	TBD	TBD
LeNet-5 on QuickDraw	TBD	TBD

MobileNet

Title	Dataset	Description	Notebooks
MobileNet-v2 on Cifar-10	TBD	TBD
MobileNet-v3 small on Cifar-10	TBD	TBD
MobileNet-v3 large on Cifar-10	TBD	TBD
MobileNet-v3 large on MNIST via Embetter	TBD	TBD

Network in Network

Title	Dataset	Description	Notebooks
Network in Network Trained on CIFAR-10	TBD	TBD

VGG

Title	Dataset	Description	Notebooks
Convolutional Neural Network VGG-16 Trained on CIFAR-10	TBD	TBD
VGG-16 Smile Classifier	CelebA	TBD
VGG-16 Dogs vs Cats Classifier	TBD	TBD
Convolutional Neural Network VGG-19	TBD	TBD

ResNet

Title	Dataset	Description	Notebooks
ResNet and Residual Blocks	MNIST	TBD
ResNet-18 Digit Classifier	MNIST	TBD
ResNet-18 Gender Classifier	CelebA	TBD
ResNet-34 Digit Classifier	MNIST	TBD
ResNet-34 Object Classifier	QuickDraw	TBD
ResNet-34 Gender Classifier	CelebA	TBD
ResNet-50 Digit Classifier	MNIST	TBD
ResNet-50 Gender Classifier	CelebA	TBD
ResNet-101 Gender Classifier	CelebA	TBD
ResNet-101	CIFAR-10	TBD
ResNet-152 Gender Classifier	CelebA	TBD

---

Transformers

Title	Dataset	Description	Notebooks
Multilabel DistilBERT	Jigsaw Toxic Comment Challenge	DistilBERT classifier fine-tuning
DistilBERT as feature extractor	IMDB movie review	DistilBERT classifier with sklearn random forest and logistic regression
DistilBERT as feature extractor using embetter	IMDB movie review	DistilBERT classifier with sklearn random forest and logistic regression using the scikit-learn embetter library
Fine-tune DistilBERT I	IMDB movie review	Fine-tune only the last 2 layers of DistilBERT classifier
Fine-tune DistilBERT II	IMDB movie review	Fine-tune the whole DistilBERT classifier

---

Ordinal Regression and Deep Learning

Please note that the following notebooks below provide reference implementations to use the respective methods. They are not performance benchmarks.

Title	Dataset	Description	Notebooks
Baseline multilayer perceptron	Cement	A baseline multilayer perceptron for classification trained with the standard cross entropy loss
CORAL multilayer perceptron	Cement	Implementation of Rank Consistent Ordinal Regression for Neural Networks with Application to Age Estimation 2020
CORN multilayer perceptron	Cement	Implementation of Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities 2022
Binary extension multilayer perceptron	Cement	Implementation of Ordinal Regression with Multiple Output CNN for Age Estimation 2016
Reformulated squared-error multilayer perceptron	Cement	Implementation of A simple squared-error reformulation for ordinal classification 2016
Class distance weighted cross-entropy loss	Cement	Implementation of Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation 2022

---

Normalization Layers

Title	Dataset	Description	Notebooks
BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier	TBD	TBD
Filter Response Normalization for Network-in-Network CIFAR-10 Classifier	TBD	TBD

Metric Learning

Title	Dataset	Description	Notebooks
Siamese Network with Multilayer Perceptrons	TBD	TBD

Autoencoders

Fully-connected Autoencoders

Title	Dataset	Description	Notebooks
Autoencoder (MNIST)	TBD	TBD
Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier	TBD	TBD

Convolutional Autoencoders

Title	Dataset	Description	Notebooks
Convolutional Autoencoder with Deconvolutions / Transposed Convolutions	TBD	TBD
Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance	TBD	TBD
Convolutional Autoencoder with Deconvolutions (without pooling operations)	TBD	TBD
Convolutional Autoencoder with Nearest-neighbor Interpolation	TBD	TBD
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA	TBD	TBD
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw	TBD	TBD

Variational Autoencoders

Title	Dataset	Description	Notebooks
Variational Autoencoder	TBD	TBD
Convolutional Variational Autoencoder	TBD	TBD

Conditional Variational Autoencoders

Title	Dataset	Description	Notebooks
Conditional Variational Autoencoder (with labels in reconstruction loss)	TBD	TBD
Conditional Variational Autoencoder (without labels in reconstruction loss)	TBD	TBD
Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss)	TBD	TBD
Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss)	TBD	TBD

Generative Adversarial Networks (GANs)

Title	Dataset	Description	Notebooks
Fully Connected GAN on MNIST	TBD	TBD
Fully Connected Wasserstein GAN on MNIST	TBD	TBD
Convolutional GAN on MNIST	TBD	TBD
Convolutional GAN on MNIST with Label Smoothing	TBD	TBD
Convolutional Wasserstein GAN on MNIST	TBD	TBD
Deep Convolutional GAN (DCGAN) on Cats and Dogs Images	TBD	TBD
Deep Convolutional GAN (DCGAN) on CelebA Face Images	TBD	TBD

Graph Neural Networks (GNNs)

Title	Dataset	Description	Notebooks
Most Basic Graph Neural Network with Gaussian Filter on MNIST	TBD	TBD
Basic Graph Neural Network with Edge Prediction on MNIST	TBD	TBD
Basic Graph Neural Network with Spectral Graph Convolution on MNIST	TBD	TBD

Recurrent Neural Networks (RNNs)

Many-to-one: Sentiment Analysis / Classification

Title	Dataset	Description	Notebooks
A simple single-layer RNN (IMDB)	TBD	TBD
A simple single-layer RNN with packed sequences to ignore padding characters (IMDB)	TBD	TBD
RNN with LSTM cells (IMDB)	TBD	TBD
RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors	TBD	TBD
RNN with LSTM cells and Own Dataset in CSV Format (IMDB)	TBD	TBD
RNN with GRU cells (IMDB)	TBD	TBD
Multilayer bi-directional RNN (IMDB)	TBD	TBD
Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News)	TBD	TBD

Many-to-Many / Sequence-to-Sequence

Title	Dataset	Description	Notebooks
A simple character RNN to generate new text (Charles Dickens)	TBD	TBD

Model Evaluation

K-Fold Cross-Validation

Title	Dataset	Description	Notebooks
Baseline CNN	MNIST	A simple baseline with traditional train/validation/test splits
K-fold with pl_cross	MNIST	A 5-fold cross-validation run using the pl_cross library

Data Augmentation

Title	Dataset	Description	Notebooks
AutoAugment & TrivialAugment for Image Data	CIFAR-10	Trains a ResNet-18 using AutoAugment and TrivialAugment

Tips and Tricks

Title	Dataset	Description	Notebooks
Cyclical Learning Rate	TBD	TBD
Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet)	TBD	TBD
Gradient Clipping (w. MLP on MNIST)	TBD	TBD

Transfer Learning

Title	Dataset	Description	Notebooks
Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10)	TBD	TBD

Visualization and Interpretation

Title	Dataset	Description	Notebooks
Vanilla Loss Gradient (wrt Inputs) Visualization (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images)	TBD	TBD
Guided Backpropagation (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images)	TBD	TBD

PyTorch Workflows and Mechanics

PyTorch Lightning Examples

Title	Dataset	Description	Notebooks
MLP in Lightning with TensorBoard -- continue training the last model	TBD	TBD
MLP in Lightning with TensorBoard -- checkpointing best model	TBD	TBD

Custom Datasets

Title	Dataset	Description	Notebooks
Custom Data Loader Example for PNG Files	TBD	TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- CSV files converted to HDF5	TBD	TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Face Images from CelebA	TBD	TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from Quickdraw	TBD	TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from the Street View House Number (SVHN) Dataset	TBD	TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Asian Face Dataset (AFAD)	TBD	TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Dating Historical Color Images	TBD	TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Fashion MNIST	TBD	TBD

Training and Preprocessing

Title	Dataset	Description	Notebooks
PyTorch DataLoader State and Nested Iterations	Toy	Explains DataLoader behavior when in nested functions
Generating Validation Set Splits	TBD	TBD
Dataloading with Pinned Memory	TBD	TBD
Standardizing Images	TBD	TBD
Image Transformation Examples	TBD	TBD
Char-RNN with Own Text File	TBD	TBD
Sentiment Classification RNN with Own CSV File	TBD	TBD

Improving Memory Efficiency

Title	Dataset	Description	Notebooks
Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10)	TBD	TBD

Parallel Computing

Title	Description	Notebooks
Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA	TBD
Distribute a Model Across Multiple GPUs with Pipeline Parallelism (VGG-16 Example)	TBD

Other

Title	Dataset	Description	Notebooks
PyTorch with and without Deterministic Behavior -- Runtime Benchmark	TBD	TBD
Sequential API and hooks	TBD	TBD
Weight Sharing Within a Layer	TBD	TBD
Plotting Live Training Performance in Jupyter Notebooks with just Matplotlib	TBD	TBD

Autograd

Title	Dataset	Description	Notebooks
Getting Gradients of an Intermediate Variable in PyTorch	TBD	TBD

TensorFlow Workflows and Mechanics

Custom Datasets

Title	Description	Notebooks
Chunking an Image Dataset for Minibatch Training using NumPy NPZ Archives	TBD
Storing an Image Dataset for Minibatch Training using HDF5	TBD
Using Input Pipelines to Read Data from TFRecords Files	TBD
Using Queue Runners to Feed Images Directly from Disk	TBD
Using TensorFlow's Dataset API	TBD

Training and Preprocessing

Title	Dataset	Description	Notebooks
Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives	TBD	TBD

Related Libraries

Title	Description	Notebooks
TorchMetrics	How do we use it, and what's the difference between .update() and .forward()?

deeplearning-models's Issues

HTTPError: HTTP Error 503: Service Unavailable

deeplearning-models/pytorch_ipynb/cnn/cnn-resnet50-mnist-dataparallel.ipynb

Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to data/MNIST/raw/train-images-idx3-ubyte.gz
0it [00:00, ?it/s]
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
<ipython-input-4-38058be550e3> in <module>
      5 # Note transforms.ToTensor() scales input images
      6 # to 0-1 range
----> 7 train_dataset = datasets.MNIST(root='data', 
      8                                train=True,
      9                                transform=transforms.ToTensor(),

MNIST download not working...

CNN Link

Convolutional Neural Network in TensorFlow link is not opening. (page not found)

Dp

how to choose the parameter num_epochs

first, thanks for your excellent project. It's very friendly for beginner.
I have noticed that on dog-vs-cats dataset the hyper parameter of num_epochs is 100 which is bigger than that on CIFAR dataset. CIFAR dataset has more data and more classes. So why we need to train out net more on this dataset?How can I choose a proper hyper parameter of num_epochs. Looking forward for your response.

Question regarding gradient checkpointing

Hello,

I am trying to understand gradient checkpointing and found your explanation in gradient-checkpointing-nin.ipynb very helpful. I cloned the repo and tried rerunning the experiments. However, I was unable to reproduce the result mentioned in your conclusion.

When I run the notebook, for the vanilla NiN, my memory consumption (current, peak) are 413527 and 154049604, with runtime 109.1s.
For the checkpointed version (segments=1) of the model, the memory consumption are 402938 and 154064699, with runtime 110.14s.
From these tests, I was not able to observe a significant improvement in memory as the notebook states (22% memory improvement with 14% runtime sacrifice).

I've tried running with multiple seeds and checkpoint segment sizes, and was not able to see a significant memory improvement either.

I'm not sure why this is and could need a bit of help. Could this be due to the size of the network is relatively small and the effects are less obvious? Or could it be the checkpointing implementation from PyTorch has changed over the years? I would appreciate it if you could provide any insight in this.

Questions on notebook wgan-1.ipynb

Hi,

I am going through the notebook wgan-1.ipynb and upon running the script as is I encounter the following error:

Traceback (most recent call last):
  File "3_wgan.py", line 162, in <module>
    real_loss = wasserstein_loss(valid, disc_pred_real)
  File "3_wgan.py", line 76, in wasserstein_loss
    return torch.mean(y_true * y_pred)
RuntimeError: The size of tensor a (128) must match the size of tensor b (100352) at non-singleton dimension 0

As a result, I updated the line in the discriminator training part to:

real_loss = wasserstein_loss(valid, disc_pred_real[:128])

so that the dimensions match.

However, with the last 96 img, and labels and the dimensions are again unmatched. I am wondering if there could be a better way to approach this?

Thank you

question about features = (features - 0.5)*2 in GAN model

Hi.
I have noticed the operation here ( features = (features - 0.5)*2) in Generative Adversarial Networks (GAN). I don't understand why we need to do this here. The mean and variance of MINIST dataset are 0.1307 and 0.3081. Can you please explain the meaning of doing so? Looking forward to your reply.

abhorrent implementation between tf and torch

In following implementations, op max_pool in pytorch uses stride 2. However, op max_pool in tensorflow uses stride 1.
I am confused about it that max_pool should sub-sample signals but in this imlementation it don't have ability.
https://github.com/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-basic.ipynb
https://github.com/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/cnn/cnn-basic.ipynb

Thanks you in advance. @rasbt

Imbalanced Classes

Hi,
Does this implementation solves any class imbalanced datasets ? Since of the features of Celeba dataset has high class imbalance, does your solution addresses this issue ?

Imbalanced Classes

Hi,

For handling imbalance, I tried to duplicate the rows of minority class in train.csv file itself. But I am getting an error while joining the paths:

Traceback (most recent call last):
File "/home/surabhi/celeba-dataset/classify_bl_aug.py", line 302, in
model_conv = train_model(model_conv, optimizer_conv, exp_lr_scheduler, num_epochs=10)
File "/home/surabhi/celeba-dataset/classify_bl_aug.py", line 207, in train_model
for inputs, labels in dataloaders[phase]:
File "/home/dataset/packages/pytorch/1.0.0/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 637, in next
return self._process_next_batch(batch)
File "/home/dataset/packages/pytorch/1.0.0/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
TypeError: Traceback (most recent call last):
File "/home/dataset/packages/pytorch/1.0.0/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/dataset/packages/pytorch/1.0.0/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/surabhi/celeba-dataset/classify_bl_aug.py", line 69, in getitem
img = Image.open(os.path.join(self.img_dir, self.img_names[index]))
File "/home/dataset/packages/python/3.7/lib/python3.7/posixpath.py", line 94, in join
genericpath._check_arg_types('join', a, *p)
File "/home/dataset/packages/python/3.7/lib/python3.7/genericpath.py", line 149, in _check_arg_types
(funcname, s.class.name)) from None
TypeError: join() argument must be str or bytes, not 'int64'

pre-trained model

thanks a lot about an amazing project. I want to ask if there are pretrained model of Resnet18 trained on celeba.

TSV to HDF5 converter on a very large dataset

Hi,
I'm trying to convert several TSV files from c4 200M dataset into HDF5 format and I based my conversion on your notebook.

The dataset is composed by 10 files, each file containing approximately 18 million records, with 2 string columns.
Given the size of the dataset, I thought that converting it to HDF5 format would give a significant benefit and would allow me to know the shape of each file and give significant performance boost in the read of chunks of the dataset.

In a first trial I converted 1 million records in about 3 minutes, however when I tried to convert all 18 million records it is taking more than 6 hours per file.
I am currently loading my tsv in the following way

def csv_to_hf5(csv_path, num_lines=1000000, chunksize=100000, columns=None):
    if columns is None:
        columns = ['input', 'labels']
    csv_path = pl.Path(csv_path)

    hdf_filename = csv_path.parent / pl.Path(csv_path).name.replace('.tsv', '.hf5')

    # suppose this is a large CSV that does not
    # fit into memory:

    # Get number of lines in the CSV file if it's on your hard drive:
    # num_lines = subprocess.check_output(['wc', '-l', in_csv])
    # num_lines = int(nlines.split()[0])
    # use 10,000 or 100,000 or so for large files

    dt = h5py.special_dtype(vlen=str)

    # this is your HDF5 database:
    with h5py.File(hdf_filename, 'w') as h5f:

        # use num_features-1 if the csv file has a column header
        dset1 = h5f.create_dataset('input',
                                   shape=(num_lines,),
                                   compression=9,
                                   dtype=dt
                                   )
        dset2 = h5f.create_dataset('labels',
                                   shape=(num_lines,),
                                   compression=9,
                                   dtype=dt
                                   )

        # change range argument from 0 -> 1 if your csv file contains a column header
        for i in tqdm(range(0, num_lines, chunksize)):
            df = pd.read_csv(csv_path,
                             sep='\t',
                             names=columns,
                             header=None,  # no header, define column header manually later
                             nrows=chunksize,  # number of rows to read at each iteration
                             skiprows=i,
                             )  # skip rows that were already read

            features = df.input.values.astype(str)
            labels = df.labels.values.astype(str)

            # use i-1 and i-1+10 if csv file has a column header
            dset1[i:i + chunksize] = features
            dset2[i:i + chunksize] = labels

where i set num_lines equal to the total lines of each file, where chunksize = 10000.

I did not expect this performance degradation, have you ever tried to use your code to convert dataset of a similar dimension?

Thanks in advance.

Is anyone contribute on Tensorflow 2.0?

Questions about the book

Dear author, thank you for offering the code.

But I can't open the ebook link https://leanpub.com/ann-and-deeplearning

Could you please send me the pdf of the book?
My email address is [email protected]

Thank you very much!

Feature Request: FCN with Vgg 16 and Resnet Backbones

Hello, Sebastian, can you please also include, FCN with Vgg16 and Resnet backbones sometime in near future. Currently I could only find pytorch implementations with Vgg16 backbones but not Resnet. Also many repos don't explain the appropriate cropping and pixel alignment with vgg16 backbone.

Perception TF v1 code savedModel graph says it has no operation

hi I am using this repo code

g = tf.Graph()
with g.as_default():
    # Graph Inputs
    features = tf.placeholder(dtype=tf.float32,
                              shape=[None, 2], name='features')
    targets = tf.placeholder(dtype=tf.float32,
                             shape=[None, 1], name='targets')

    # Model Parameters
    weights = tf.Variable(tf.zeros(shape=[2, 1],
                                   dtype=tf.float32), name='weights')
    bias = tf.Variable([[0.]], dtype=tf.float32, name='bias')

    # Forward Pass
    linear = tf.add(tf.matmul(features, weights), bias, name='linear')
    ones = tf.ones(shape=tf.shape(linear))
    zeros = tf.zeros(shape=tf.shape(linear))
    prediction = tf.where(condition=tf.less(linear, 0.),
                          x=zeros,
                          y=ones,
                          name='prediction')

    # Backward Pass
    errors = targets - prediction
    weight_update = tf.assign_add(weights,
                                  tf.reshape(errors * features, (2, 1)),
                                  name='weight_update')
    bias_update = tf.assign_add(bias, errors,
                                name='bias_update')

    train = tf.group(weight_update, bias_update, name='train')

    saver = tf.train.Saver(name='saver')

and save it using

inputs = dict([(features.name, features)])
outputs = dict([(prediction.name, prediction)])
tf.saved_model.simple_save(sess, "my_path", inputs, outputs)

and I can use saved_model_cli to see the model, following is part of it

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['features:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 2)
        name: features:0

but when I use TF2 tf.keras.model.load_model("my_path") it raise error KeyError: "The name 'features:0' refers to a Tensor which does not exist. The operation, 'features', does not exist in the graph.", using java api raise the similar error.

Could this save to savedModel? How should I do it correctly?

a problem about LeNet-5 Classifier

Hi,
The model of LeNet-5 contains no nonlinearity. Is it wrong?

test_dataset with train=True in cnn-densenet121-mnist.ipynb

test_dataset = datasets.MNIST(root='data', 
                              train=True, 
                              transform=transforms.ToTensor(),
                              download=True)

Validation ACC: 98.50%
Test ACC: 99.91%

https://github.com/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-densenet121-mnist.ipynb

In cnn-vgg16.ipynb, its acc has remained unchanged at 10%!

The results are as follows.

# Epoch: 001/010 | Train: 10.000% | Loss: 2.520%
# Time elapsed: 2.24 min
# Epoch: 010/010 | Train: 10.000% | Loss: 2.303%
# Time elapsed: 22.42 min
# Total Training Time: 22.42 min
# Test accuracy: 10.00%

I have tried to increase the learning rate by adding Dropout and nn.Adaptive eavgpool2d, but acc has remained unchanged!
Thanks!

Convolutional Conditional Variational Autoencoder with label in recon loss is bad

Hi， I check your Notebook but， the preformance is very bad right？ why？

The out channel number in cnn_basic pytorch

28x28x1 => 28x28x4

    self.conv_1 = torch.nn.Conv2d(in_channels=1,
                                  out_channels=8,

The out_channels = 8 shouldn't it become 28x28x8

Question about conditional autoencoder

Hi sebastian,
I have some questions about CVAE,hope you can help me understand
So,rather than training CVAE on images,I am trying to train it on a list of numpy arrays,each array contains 45 parameters.
As for my condition variable it consists of 33 parameters .
in all the training X's total size is 1500x43 and the condition variable's size is 33x3
Any ideas on what modifications I can do to run CVAE on my example.Basically I am trying to generate new values for each of the 45 parameters given the values in the condition variable.
Also,will a CVAE be a suitable choice for my case.

No file of "pytorch_ipynb/mlp/mlp-sequential.ipynb"

In the folder of pytorch_ipynb/mlp,I can't find the file "Sequential API and hooks [PyTorch]“.

missing data folder

I tried to run the perceptron model locally after cloning. But the data folder is missing in the repo.
I got this error : OSError: ../../ch02_perceptron/perceptron_toydata.txt not found.

multi label

Hello,

I appreciate your work... I would like to upgrade it from binary labels to multi labels.

class CelebaDataset(Dataset):
    """Custom Dataset for loading CelebA face images"""

    def __init__(self, csv_path, img_dir, transform=None):
    
        df = pd.read_csv(csv_path, index_col=0)
        self.img_dir = img_dir
        self.csv_path = csv_path
        self.img_names = df.index.values
        self.y = df['Male'].values # <-- this needs to be changed  with other labels...
        self.transform = transform

What to do if I want to change it to several classes?

NameError: name 'custom_where' is not defined

Hello,I discovered the following problem in pycharm
deeplearning-models/pytorch_ipynb/basic-ml/perceptron.ipynb

Traceback (most recent call last):
  File "/home/aibc/Desktop/DL/pytorch_ML/traditional_ml/preceptron/preceptron.py", line 99, in <module>
    ppn.train(X_train_tensor, y_train_tensor, epochs=5)
  File "/home/aibc/Desktop/DL/pytorch_ML/traditional_ml/preceptron/preceptron.py", line 84, in train
    errors = self.backward(x[i].view(1, self.num_features), y[i]).view(-1)
  File "/home/aibc/Desktop/DL/pytorch_ML/traditional_ml/preceptron/preceptron.py", line 75, in backward
    predictions = self.forward(x)
  File "/home/aibc/Desktop/DL/pytorch_ML/traditional_ml/preceptron/preceptron.py", line 71, in forward
    predictions = custom_where(linear > 0., 1, 0).float()
NameError: name 'custom_where' is not defined

I looked up the definition of the function, but nothing came of it.
I hope you can help me. Thank you

Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News)

In this notebook, while defining the model you didn't apply self.fc2()
So the shape of the returned tensor is (128, 64) instead of (128, 4).
But the interesting part is that it still achieves great accuracy. Now I'm just wondering why it is working.
Also, model(text, text_lengths) returns a tensor of size (128, 64), so why are we calling .squeeze(1) on that tensor?
Since the second dimension is not 1, the tensor shape remains the same. Is there any edge case we are dealing with?

Understanding the models

Thank you SO MUCH for organizing the ever growing corpus. HOWEVER, it would be nice if there were an algorithm that would help me choose the right one. For example, if I wanted to do reverse style transfer (convert a monet painting into a photograph) Which model should I use? And obviously I would first need a training set (which I could generate by processing a billion images from google with a generic style transfer app).