Code for distillation part

[facebookresearch]

Code for distillation part

[fmassa]

Hi,

We are working on preparing the release for the distillation part in https://github.com/fmassa/deit/tree/distillation, and we will be opening a PR once we finish validating that the refactored code works, which should be later this week.

[ZhangChao1993]

Hi,

We are working on preparing the release for the distillation part in https://github.com/fmassa/deit/tree/distillation, and we will be opening a PR once we finish validating that the refactored code works, which should be later this week.

Thanks for your great work! By the way, can you share us fine-tuing code? I tried 384 for pretrained model and cannot work!
Others need this too! link

[fmassa]

Hi,

Code for distillation has been merged into master, and the code for finetuning is in #43 and will be merged soon. Once they are merged I'll upload the weights for the pre-trained models.

[fmassa]

Hi,

We've merged the code for distillation and finetuning in #42 and #43, and added the pre-trained weights in #50.
Given that those are now available, I'm closing this issue but let us know if you have any further questions