xuefuzhao/awesome-mixture-of-experts

A collection of AWESOME things about mixture-of-experts

Hello there! This is the Github page of Fuzhao Xue.

• 📖 Pursuing PhD in CS at National University of Singapore.
• 🎓 Conducting research on Large Langauge Model Scaling, Conditional Computation and Adaptive Computation.
• 💡 Have a wide interest on applications of Machine Learning, Natural Language Processing, HIgh Performance Computing, ...
• 👀 Open to collaborating on new projects.
• 💬 Visit my homepage for more information.

awesome-mixture-of-experts's Issues

FasterMoE paper on PPoPP'22

There is a new distributed system FasterMoE: modeling and optimizing training of large-scale dynamic pre-trained models published on PPoPP'22. As long as you have already put BaGuaLu in your list, please also kindly consider including this paper.

FYI, we have also indicated this paper collections at FastMoE's homepage