ryoungj/optdom

[ICLR'22] Self-supervised learning optimally robust representations for domain shift.

optdom's Issues

Missing PACS.zip file

https://drive.google.com/uc?id=0B6x7gtvErXgfbF9CSk53UkRxVzg

Why there is a performance gap between the CAD in this repo and the Domainbed repo

Hi @ryoungj! Thanks for this outstanding work. I really appreciate your public code.

Recently, I am trying to reproduce the results of CAD (based on ResNet-50 backbone) in your paper. But I found a performance gap between this repo (83.5) and the domainbed repo (79.5) in PACS. In detail, for both repositories, I swept 40 hyper-params already. And in this repo, I use the run_sweep_e2e_domainbed.sh.

My questions are

• (1) Could you help me clarify why there is a performance gap between these repositories? and which one should I use?
• (2) The reported "CAD" results in your paper are all for the CAD instead of CondCAD. Is this right?
• (3) By the way, if my understanding is correct, the results of CAD based on ResNet-50 are reported in Tab. 4 and 5 in your paper?

Pls correct me if there is something wrong. Thank you so much.

About CLIP S results

Hello @ryoungj,

first of all, thank you for sharing your code! This is an amazing work!

I read your paper and I would start to replicate the results that you reported in Table 1.
In particular, I would replicate "CLIP S" (4th row).
If I correctly understood these numbers are obtained using as feature extractor the pre-trained CLIP model (Resnet-50). With the source features extracted from this pre-trained model, an MLP is trained with a supervised contrastive loss function.

So my questions are:
1 - Did I correctly understand CLIP S meaning?
2 - Why did you use a supervised contrastive loss function instead of a standard cross-entropy loss?
3 - How can I replicate these numbers using the code that you shared?

Thank you.