alphadl / awesome-knowledge-distillation-of-llms Goto Github PK
View Code? Open in Web Editor NEWThis project forked from tebmer/awesome-knowledge-distillation-of-llms
This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". It includes KD Algorithms (Labeling, Self-Knowledge, Self-Rewarding, SFT, etc.) leveraging data augmentation or synthesis for getting knowledge, Skill Distillation (e.g. instruction following), and specialized applications (law, healthcare).