soapisnotfat/pytorch-cifar10

pytorch and cifar10

pytorch-cifar10

Personal practice on CIFAR10 with PyTorch
Inspired by pytorch-cifar by kuangliu.

Introduction

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.

Requirement

• python3.6
• numpy
• pytorch 0.4.0
• torchvision 0.2.0

Usage

python3 main.py

optional arguments:

--lr                default=1e-3    learning rate
--epoch             default=200     number of epochs tp train for
--trainBatchSize    default=100     training batch size
--testBatchSize     default=100     test batch size

Configs

200 epochs for each run-through,
500 batches for each training epoch,
100 batches for each validating epoch,
100 images for each training and validating batch

Learning Rate

1e-3 for [1,74] epochs
5e-4 for [75,149] epochs
2.5e-4 for [150,200) epochs

Result

Models	Accuracy	Comments
LeNet	67.52%	- - - -
Alexnet	74.74%	Result is far away from my expectation (5%+). Reasons might be inappropriate modification to fit dataset(32x32 images).
VGG11	87.48%	- - - -
VGG13	90.17%	- - - -
VGG16	TBD	- - - -
VGG19	TBD	- - - -
GoogleNet	92.57%	- - - -
ResNet18	TBD	- - - -
ResNet34	TBD	- - - -
ResNet50	TBD	- - - -
ResNet101	TBD	- - - -
ResNet152	TBD	- - - -
DenseNet121	TBD	- - - -
DenseNet161	TBD	- - - -
DenseNet169	TBD	- - - -
DenseNet201	TBD	- - - -

pytorch-cifar10's Issues

resnet implementation error

In the resnet.py, the following codes should be revised.

if self.downsample is not None:
residual = self.downsample(x)

it should be

if self.downsample is not None:
residual = self.downsample(residual)

confirming bad accuracy of alexnet on cifar

I just tried alexnet on cifar and got 69.09% on test set even with cv, although I trained for 100 epochs standard lr for adam (i.e. no mulstistep lr), batch size of 1024 for training and 64 for final test set evaluation.

change "%d/200" to "%d/%d" for correct epochs preview

https://github.com/icpm/pytorch-cifar10/blob/1e49822683bf0800a3f4f7f0df778cb5ba6a87fe/main.py#L141

Googlenet has no attribute to

While I try to execute the code I get an error, googlenet has no attribute to.

self.scheduler.step() in () should not have epoch

        self.scheduler.step() 

not self.scheduler.step(epoch)

Reduce the training set, leading to overfitting

When I only use 10,000 training data, the loss function of the test dataset of each network has been rising instead of falling, which will lead to very serious overfitting.Training accuracy rate is 100%, test accuracy rate is 10%. How can I solve this problem?

can pytorch 1.0 run it?

hi!
I install pytorch 1.0 , when I run main.py, it says:
No module named 'torchvision'.
if I conda install torchvison, it seems to install pytorch 0.4.0. Does pytorch1.0 not contain torchvison?

Alexnet high accuracy

I trained the "Alexnet" in another implementation and got very high accuracy, around 85%.

The main differences between the two implementations are,

1. random cropping in train_transform "transforms.RandomCrop(32, padding=4)",
2. SGD optimizer with step-wise learning rate [100 150 180] (total 200 epochs) and an initial learning rate 0.1.

Is this high accuracy possible?

RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:383

OS: Ubuntu
pytorch: 1.1.0
cuda: 9.0
cudnn:7.6
GPU: RTX 2080

I got errors below:
runpy.run_path(target, run_name='__main__') File "/usr/lib/python3.6/runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "/usr/lib/python3.6/runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/gaoya/Downloads/pytorch-cifar10-master/main.py", line 155, in <module> main() File "/home/gaoya/Downloads/pytorch-cifar10-master/main.py", line 27, in main solver.run() File "/home/gaoya/Downloads/pytorch-cifar10-master/main.py", line 145, in run train_result = self.train() File "/home/gaoya/Downloads/pytorch-cifar10-master/main.py", line 95, in train output = self.model(data) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/home/gaoya/Downloads/pytorch-cifar10-master/models/AlexNet.py", line 38, in forward x = self.features(x) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py", line 338, in forward self.padding, self.dilation, self.groups) RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:383