linusericsson/ssl-transfer

Official code for the CVPR 2021 paper "How Well Do Self-Supervised Models Transfer?"

ssl-transfer's Issues

Images are repeated within ```ssl-transfer/caltech101-splits/train.txt``` file.

Images are repeated within ssl-transfer/caltech101-splits/train.txt file. For example: example-1, example-2, example-3 and there are many more. Total unique element in the txt file is 2416 instead of 3060.

Reproduction Difference

Hello!

First, I want to acknowledge the effort you have done testing all those models on different datasets. Thanks for that.

I am trying to reproduce InfoMin model performance on the DTD dataset.
However, I am achieving a score that is below the reported score by 10 degrees of performance.

Any thoughts?

Data download?

Hi, thanks for your great work. I want to ask that how can I quickly downloader all the data (cars, flowers, SUN and so on).　Would you have any scripts? Thanks a lot for your help.

Object detection transfer supervised baseline replication issues

Hi,

I am having troubles reproducing your supervised baseline for object detection with frozen backbone as a baseline for my own experiments.

I am running detectron2 with ResNet50 torchvision weights converted to detectron2 with the official script and freeze the backbone. I have also tested the MSRA weights normally used by detectron2.

Results:

AP AP50 AP75
44.25 77.14 44.07 (mine, torchvision weights)
46.90 78.27 49.19 (mine, MSRA weights)
51.99 81.53 56.21 (yours, table 3 frozen backbone)

Could you share

• your detectron2 configuration files for your frozen and fine-tune experiments?
• how you are freezing the backbone? Are you using the detectron2 FREEZE_AT config parameter?

I am running the detectron2 VOC configuration modified with the settings in section 4.3 from your paper and freeze the backbone using FREEZE_AT:

_BASE_: "censored-path/detectron2/configs/PascalVOC-Detection/faster_rcnn_R_50_FPN.yaml"
MODEL:
  BACKBONE:
    FREEZE_AT: 5    # freeze all resnet stages
  WEIGHTS: "censored-path/models/r50-imagenet1k-torchvision.pkl"
  PIXEL_MEAN: [123.675, 116.280, 103.530]
  PIXEL_STD: [58.395, 57.120, 57.375]
  RESNETS:
    DEPTH: 50
    STRIDE_IN_1X1: False
INPUT:
  FORMAT: "RGB"
SOLVER:
  STEPS: (96000, 128000)
  MAX_ITER: 144000
  WARMUP_ITERS: 100
  BASE_LR: 0.0025
  IMS_PER_BATCH: 2
  CHECKPOINT_PERIOD: 24000
OUTPUT_DIR: censored-path

Best,
Constantin

Best Hyperparameters

Hello,
I just wonder where I can find the best hyperparameters "C" according to your experiments.
For different datasets I mean.
Regards,

Encoder model for Semantic-Segmentation

Hello!

Thanks for the great repo!
I had some trouble in using models for semantic-segmentation. It seems like the MIT-Seg repo has a custom resnet model. I get the following error when I try to run python train.py --gpus 0,1 --cfg selfsupconfig/moco-v2.yaml :

Loading weights for net_encoder
Traceback (most recent call last):
  File "train.py", line 273, in <module>
    main(cfg, gpus)
  File "train.py", line 144, in main
    net_encoder = ModelBuilder.build_encoder(
  File "/cis/home/ashah/Code/ssl-transfer/semantic-segmentation/mit_semseg/models/models.py", line 108, in build_encoder
    net_encoder.load_state_dict(
  File "/cis/home/ashah/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1044, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Resnet:
        size mismatch for conv1.weight: copying a param with shape torch.Size([64, 3, 7, 7]) from checkpoint, the shape in current model is torch.Size([64, 3, 3, 3]).
        size mismatch for layer1.0.conv1.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([64, 128, 1, 1]).
        size mismatch for layer1.0.downsample.0.weight: copying a param with shape torch.Size([256, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 128, 1, 1]).

Is this expected ? Did you skip loading weights for these layers when finetuning ?
Thanks!

Different reproduction results for SimCLR, BYOL

I notice that your SimCLR and BYOL results do not match the results in the original papers (I suspect many others do not match too). Is your reproduction different from the original papers? The differences are so big that I doubt it can be due to randomness. For example, the SimCLR result on Aircraft is 50.3 in the original paper but 44.90 in your paper.

NYU v2 all_rgb and surfacenormal_metadata sources

Hi,
Nice work! I am using your codes to evaluate the surface normal estimation performance of my pre-trained backbones. I encounter some issues when formatting the dataset. In make_SN_labels.py, it seems to require all_rgb and surfacenormal_metadata to create metadata_son and restructure images. I could not find an instruction in the repo to prepare these files. NYUv2 website only provides a .mat file as the dataset from which I need to extract images by myself. Although I can do these things based on my conjecture, I am not sure if I could do it properly. Could you please help to provide a link where you download those sources? Thank you very much!

Caltech 101 data split?

Great work !
Could you please also provide the data split of the Caltech-101 dataset to reproduce the results in the paper?
Thanks a lot.