Clarification on Fewshot results in the paper

[zhangyuanhan-ai]

Clarification on Fewshot results in the paper

[ZhangYuanhan-AI]

Thank you for the catch.

In food101 (shot 4,8,16), we find that the searched subset performs much better (in shot 8, 71.5 on average) if it inherits weights from its supernet rather than retraining. So we report this performance on the paper. But, in other datasets, we do not find this phenomenon

[muqeeth]

Thanks for clarifying @davidzhangyuanhan. Can you help with replicating 71.5 on food 101? I am planning to do the following:

1. Train supernet on food 101 dataset using the following command.
   python supernet_train_prompt.py --data-path=./data/${DATASET} --data-set=${DATASET}-FS --cfg=${CONFIG} --resume=${CKPT} --output_dir=./saves/few-shot_${DATASET}_shot-${SHOT}_seed-${SEED}_lr-${LR}_wd-${WEIGHT_DECAY}_supernoah --batch-size=64 --lr=${LR} --epochs=300 --weight-decay=${WEIGHT_DECAY} --few-shot-seed=${SEED} --few-shot-shot=${SHOT} --launcher="none"

where CKPT=pretrained vit16, CONFIG=./experiments/NOAH/supernet/supernet-B_prompt.yaml, LR=5e-4, WEIGHT_DECAY=0.0001, SHOT=8, DATASET=food-101

2. Run eval using the optimal architecture you already found and written in config with using above trained weights as resume checkpoint.

I am guessing I need not do search since the optimal architecture configuration is already provided in the config.

Is that right? or should I retrain the architecture with weights initialized from the supernet trained above?

Please let me know if I am doing anything wrong.

[ZhangYuanhan-AI]

Yes, you are correct.

I'd like to share with you the (seed-0) checkpoint of the supernet and the evaluation log for reference.

[muqeeth]

Thank you!!