sutd-visual-computing-group / ls-kd-compatibility Goto Github PK
View Code? Open in Web Editor NEW[ICML 2022] This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). We suggest to use an LS-trained teacher with a low-temperature transfer to render high performance students.
Home Page: https://keshik6.github.io/revisiting-ls-kd-compatibility/
License: MIT License