nickhoernle/semantic_loss

Repository for the semantic WMI loss

semantic_loss's Issues

Reproduce the results for CIFAR-100

Dear authors,

I tried to run the CIFAR-100 experiment with the provided command run_image_experiments.py cifar100 --layers=10 --widen_factor=1 run but the accuracy is as follows. (I tried this command again but the accuracy is even lower.)

...
Train [200/200]: Loss 1.046	 Acc 69.507	AccSC 79.418
Test  [200/200]: Loss 1.646	 Acc 58.16	AccSC 70.78
======== TESTING ON UNSEEN DATA =========
======== USE FINAL MODEL =========
Test  [1/200]: Loss 1.668	 Acc 58.43	AccSC 71.07
Test  [200/200]: Loss 1.668	 Acc 58.43	AccSC 71.07
Final Model accuracy ====> 2.739713430404663
======== USE BEST MODEL =========
Test  [1/200]: Loss 1.67	 Acc 58.42	AccSC 71.14
Test  [200/200]: Loss 1.67	 Acc 58.42	AccSC 71.14
Final Model accuracy ====> 1.0581140518188477

Could you help me with the following questions? Thank you!

1. How can I reproduce the results in the paper? For example, I cannot get the 74.4 ± (0.2) class accuracy and 85.4 ± (0.3) super-class accuracy, and it's not clear how to print out the constraint satisfaction accuracy.
2. How can I interpret the Final Model accuracy ====> 1.0581140518188477 in the above result?
3. How can I run the baseline for comparison?
4. The notebook https://github.com/NickHoernle/semantic_loss/blob/master/notebooks/cifar100_experiment.ipynb only prints out some pre-defined numbers. When would this notebook be updated?

Your help and time are greatly appreciated!

Best,
Zhun