a Ready-to-use PyTorch Extension of Unofficial CutMix Implementations.
This re-implementation is improved in some parts,
- Fixing issue #1 in the original repository
- issue #3 : Random crop regions are randomly chosen, even within the same batch.
- issue #4 : Different lambda values(sizes of crop regions) are randomly chosen, even within the same batch.
- Images to be cropped are randomly chosen in the whole dataset. Original implementation selects images only inside the same batch(shuffling).
- Easy to install and use on your existing project.
Hence, there may be slightly-improved training results also.
- python3
- torch >= 1.1.0
This repository is pip-installable,
$ pip install git+https://github.com/ildoonet/cutmix
or you can copy 'cutmix' folder to your project to use it.
Our CutMix
is inhereted from the PyTorch Dataset class so you can wrap your own dataset(eg. cifar10, imagenet, ...). Also we provide CutMixCrossEntropyLoss
, soft version of cross-entropy loss, which accept soft-labels required by cutmix.
from cutmix.cutmix import CutMix
from cutmix.utils import CutMixCrossEntropyLoss
...
dataset = datasets.CIFAR100(args.cifarpath, train=True, download=True, transform=transform_train)
dataset = CutMix(dataset, num_class=100, beta=1.0, prob=0.5, num_mix=2) # this is paper's original setting for cifar.
...
criterion = CutMixCrossEntropyLoss(True)
for _ in range(num_epoch):
for input, target in loader: # input is cutmixed image's normalized tensor and target is soft-label which made by mixing 2 or more labels.
output = model(input)
loss = criterion(output, target)
loss.backward()
optimizer.step()
optimizer.zero_grad()
Top-1 Error(@300epoch) | Top-1 Error(Best) | Model File | |
---|---|---|---|
Paper's Reported Result | N/A | 13.81 | N/A |
Our Re-implementation | 13.68 | 13.15 | Download(12.88) |
We ran 6 indenpendent experiments with our re-implemented codes and got top-1 errors of 13.09, 13.29, 13.27, 13.24, 13.15 and 12.88, using below command. (Converged at 300epoch with the top-1 errors of 13.55, 13.66, 13.95, 13.9, 13.8 and 13.32.)
$ python train.py -c conf/cifar100_pyramid200.yaml
Top-1 Error(@300epoch) | Top-1 Error(Best) | Model File | ||
---|---|---|---|---|
ResNet18 | Reported Result \wo CutMix | N/A | 30.43 | |
Ours | 29.674 | 29.56 | Download | |
ResNet34 | Reported Result \wo CutMix | N/A | 26.456 | |
Ours | 24.7 | 24.57 | Download | |
ResNet50 | Paper's Reported Result | N/A | 21.4 | N/A |
Author's Code(Our Re-run) | 21.768 | 21.586 | N/A | |
Our Re-implementation | 21.524 | 21.340 | Download(21.25) |
$ python train.py -c conf/imagenet_resnet50.yaml
We ran 5 independent experiments on ResNet50.
-
Author's codes
- 300epoch : 21.762, 21.614, 21.762, 21.644, 21.810
- best : 21.56, 21.556, 21.666, 21.498, 21.648
-
Our Re-implementation
- 300epoch : 21.53, 21.408, 21.55, 21.4, 21.73
- best : 21.392, 21.328, 21.386, 21.256, 21.34
- Official
- Paper : https://arxiv.org/abs/1905.04899
- Implementation : https://github.com/clovaai/CutMix-PyTorch
- ShakeDrop