AttentioNCE integrates the attention mechanism into contrastive learning to guide the model's attention towards high-quality samples while disregarding noisy ones. Consequently, AttentioNCE constructs a variational lower bound for an ideal contrastive loss, offering a worst-case guarantee for maximum likelihood estimation under noisy conditions.
--d_pos
: scaling factor for positive samples.
--d_neg
: scaling factor for negative samples.
For instance, run the following command to train an embedding on different datasets.
python main.py --dataset_name 'cifar10' --d_pos 1 --d_neg 10
python main.py --dataset_name 'stl10' --d_pos 2 --d_neg 0.5
python main.py --dataset_name 'cifar100' --d_pos 1 --d_neg 10
python main.py --dataset_name 'tinyImageNet' --d_pos 4 --d_neg 1
The model is evaluated by training a linear classifier after fixing the learned embedding.
path flags:
--model_path
: choose the model for evaluation.
python linear.py --dataset_name 'stl10' --model_path '../results/stl10/stl10_SimCLR_4model_256_400_2.0_0.5.pth'
Method | Arch | Epoch | Batch Size | Accuracy(%) | Download | ||
---|---|---|---|---|---|---|---|
SimCLR | - | - | ResNet50 | 400 | 256 | 91.12 | model |
AttentioNCE | 1 | 10 | ResNet50 | 200 | 256 | 92.42 | model |
AttentioNCE | 1 | 10 | ResNet50 | 400 | 256 | 93.08 | model |
Method | Arch | Epoch | Batch Size | Accuracy(%) | Download | ||
---|---|---|---|---|---|---|---|
SimCLR | - | - | ResNet50 | 400 | 256 | 80.15 | model |
AttentioNCE | 2 | 0.5 | ResNet50 | 200 | 256 | 87.12 | model |
AttentioNCE | 2 | 0.5 | ResNet50 | 400 | 256 | 89.45 | model |
Method | Arch | Epoch | Batch Size | Accuracy(%) | Download | ||
---|---|---|---|---|---|---|---|
SimCLR | - | - | ResNet50 | 400 | 256 | 66.55 | model |
AttentioNCE | 1 | 10 | ResNet50 | 200 | 256 | 69.78 | model |
AttentioNCE | 1 | 10 | ResNet50 | 400 | 256 | 70.23 | model |
Method | Arch | Latent Dim | Batch Size | Accuracy(%) | Download | ||
---|---|---|---|---|---|---|---|
SimCLR | - | - | ResNet50 | 400 | 256 | 53.40 | model |
BCL | 4 | 1 | ResNet50 | 200 | 256 | 56.58 | model |
BCL | 4 | 1 | ResNet50 | 400 | 256 | 58.61 | model |
Part of this code is credited to SimCLR(ICML 2020), DCL(NeurIPS 2020) and HCL(ICLR 2021).