Code to recreate STL10 experiments

[hobbitlong]

Code to recreate STL10 experiments

[HobbitLong]

An example of transformation looks like:

mean = [(0 + 100) / 2,
        (-86.183 + 98.233) / 2,
        (-107.857 + 94.478) / 2]
std = [(100 - 0) / 2,
       (86.183 + 98.233) / 2,
       (107.857 + 94.478) / 2]
train_transform = transforms.Compose([
    transforms.RandomCrop(64),
    # transforms.RandomResizedCrop(64, scale=(0.08, 1)), 
    transforms.RandomHorizontalFlip(), 
    RGB2Lab(),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
])
test_transform = transforms.Compose([
    transforms.CenterCrop(64),
    RGB2Lab(),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
])

Using RandomCrop without resize should be able to reproduce the results in the paper. RandomResizedCrop could further improve the results, though.

[HobbitLong]

I enjoyed reading the paper and thanks for uploading the code.

Quick question - would it be possible to also upload the scripts to run the STL-10 eval?

Thanks!

Hi, thanks for your interest!

I plan to but would not do it now. If you cannot wait, simply modifying the train_CMC.py and following the hyperparameters in the paper will work.

[macaodha]

If possible it would also be great if you could upload your trained models. Thanks!

BTW are the following augmentations accurate for STL10 i.e. no color transformations and resize (as opposed to center crop) for the test loader?

im_size = 64
norm_vals = [(0.5, 0.5, 0.5), (0.5, 0.5, 0.5)]

transform_train = transforms.Compose([
    transforms.RandomResizedCrop(size=im_size, scale=(0.2,1.0)),    
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize(norm_vals[0], norm_vals[1]),
])

transform_test = transforms.Compose([
    transforms.Resize(im_size),
    transforms.ToTensor(),
    transforms.Normalize(norm_vals[0], norm_vals[1]),
])

[macaodha]

Based on your comments, I managed to get within 1% of the results in the paper for CMC NCE linear fc7 for STL10. Would you mind letting me know what batch size, momentum, and weight_decay you used for the self-supervised training and also if there were any changes to the existing code when training the linear classifier.

Thanks!

[HobbitLong]

Hi,

For self-supervised NCE CMC stage on STL-10:
batch_size = 64 or 128 (I used the latter, both delivers similar accuracy), momentum = 0.9, weight_decay = 1e-4.

For Linear Classifier on STL-10, I used Adam following the original Deep Infomax. The detailed setting is:
lr = 1e-3, betas = (0.5, 0.999), weight_decay = 0. Remember to decay the learning rate at epoch 60, 80 by a factor of 5. And 100 epochs in total.

Let me know if there are other questions.

[HobbitLong]

Seems it's not active now, I am closing it. Feel free to reopen it.