Topic: mixture-of-experts Goto Github

👇 Here are 100 public repositories matching this topic...

adamg012 / moe-paper-models
mixture-of-experts,A sumary of MoE experimental setups across a number of different papers.
User: adamg012
awesome-list language-model machine-learning machine-translation mixture-of-experts moe papers transformer transformers

amazaspshumik / mixture-models
mixture-of-experts,Hierarchical Mixture of Experts,Mixture Density Neural Network
User: amazaspshumik
hierarchical-mixtures-of-experts mixture-of-experts

amazaspshumik / mtlearn
mixture-of-experts,Multi-Task Learning package built with tensorflow 2 (Multi-Gate Mixture of Experts, Cross-Stitch, Ucertainty Weighting)
User: amazaspshumik
cross-stitch experts kdd2018 keras mixture-of-experts multi-task-learning multitask-learning papers-reproduced papers-with-code recsys2019 tensorflow2

avisoori1x / makemoe
mixture-of-experts,From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)
User: avisoori1x
large-language-models llm mixture-of-experts deep-learning neural-networks pytorch pytorch-implementation

borealisai / mmoeex-mtl
mixture-of-experts,PyTorch Implementation of the Multi-gate Mixture-of-Experts with Exclusivity (MMoEEx)
Organization: borealisai
mixture-of-experts multi-task-learning pytorch

bwconrad / soft-moe
mixture-of-experts,PyTorch implementation of "From Sparse to Soft Mixtures of Experts"
User: bwconrad
computer-vision machine-learning mixture-of-experts transformer vision-transformer pytorch

checkstep / mole-stance
mixture-of-experts,MoLE: Cross-Domain Label-Adaptive Stance Detection
Organization: checkstep
Home Page: https://arxiv.org/abs/2104.07467
nlp artificial-intelligence ai stance-detection stance cross-domain mole label-adaptation stance-classification mixture-of-experts

davidmrau / mixture-of-experts
mixture-of-experts,PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538
User: davidmrau
moe mixture-of-experts sparsely-gated-mixture-of-experts pytorch re-implementation

drawbridge / keras-mmoe
mixture-of-experts,A TensorFlow Keras implementation of "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts" (KDD 2018)
Organization: drawbridge
machine-learning deep-learning data-science deep-neural-networks kdd2018 keras tensorflow multi-task-learning mixture-of-experts

dsy109 / mixtools
mixture-of-experts,Tools for Analyzing Finite Mixture Models
User: dsy109
cran r mixture-models semiparametric-regression mixture-of-experts

dvmazur / mixtral-offloading
mixture-of-experts,Run Mixtral-8x7B models in Colab or consumer desktops
User: dvmazur
colab-notebook deep-learning google-colab language-model llm mixture-of-experts offloading pytorch quantization

eduardzamfir / seemoredetails
mixture-of-experts,Repository for our paper "See More Details: Efficient Image Super-Resolution by Experts Mining"
User: eduardzamfir
Home Page: https://eduardzamfir.github.io/seemore
computer-vision efficiency low-rank-matrix-decomposition mixture-of-experts super-resolution low-level-vision icml-2024

efeslab / fiddler
mixture-of-experts,Fast Inference of MoE Models with CPU-GPU Orchestration
Organization: efeslab
Home Page: https://arxiv.org/abs/2402.07033
llm llm-inference mixtral-8x7b mixture-of-experts local-inference

fkodom / soft-mixture-of-experts
mixture-of-experts,PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
User: fkodom
deep-learning mixture-of-experts pytorch transformer vision-transformer

gaozhitong / mose-auseg
mixture-of-experts,The official code repo for the paper "Mixture of Stochastic Experts for Modeling Aleatoric Uncertainty in Segmentation". (ICLR 2023)
User: gaozhitong
Home Page: https://arxiv.org/pdf/2212.07328.pdf
aleatoric-uncertainty generative-model mixture-of-experts optimal-transport semantic-segmentation uncertainty-estimation stochastic-segmentation

hanyas / mimo
mixture-of-experts,A toolbox for inference of mixture models
User: hanyas
hierarchical-models mixture-models inference variational-inference markov-chain-monte-carlo vi mcmc gibbs-sampling clustering regression

hltchkust / moel
mixture-of-experts,MoEL: Mixture of Empathetic Listeners
Organization: hltchkust
transformer chatbot empathy dialogue-systems mixture-of-experts dialogue-generation transformer-pytorch

inferflow / inferflow
mixture-of-experts,Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
User: inferflow
llama2 llamacpp llm-inference model-quantization multi-gpu-inference mixture-of-experts moe gemma falcon minicpm

james-oldfield / mumoe
mixture-of-experts,[arXiv'24] Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization
User: james-oldfield
Home Page: http://james-oldfield.github.io/MMoE
mixture-of-experts moe

koayon / awesome-adaptive-computation
mixture-of-experts,A curated reading list of research in Adaptive Computation, Dynamic Compute & Mixture of Experts (MoE).
User: koayon
adaptive-computation computer-vision machine-learning mixture-of-experts pytorch tensorflow transformers nlp

kyegomez / limoe
mixture-of-experts,Implementation of the "the first large-scale multimodal mixture of experts models." from the paper: "Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts"
User: kyegomez
Home Page: https://discord.gg/47ENfJQjMq
ai artificial-intelligence machine-learning mixture-of-experts ml moe pytorch swarms tensorflow

kyegomez / switchtransformers
mixture-of-experts,Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"
User: kyegomez
Home Page: https://discord.gg/GYbXvDGevY
ai gpt4 llama mixture-of-experts ml moe multi-modal mixture-model mixture-of-models

learning-at-home / hivemind
mixture-of-experts,Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
Organization: learning-at-home
deep-learning pytorch volunteer-computing mixture-of-experts distributed-training distributed-systems asynchronous-programming asyncio dht hivemind

leeroo-ai / leeroo_orchestrator
mixture-of-experts,The implementation of "Leeroo Orchestrator: Elevating LLMs Performance Through Model Integration"
Organization: leeroo-ai
Home Page: https://www.leeroo.com
ai large-language-models llm mixture-of-experts nlp orchestration-of-experts cost efficiency gpt4 opensource

leeroo-ai / mergoo
mixture-of-experts,A library for easily merging multiple LLM experts, and efficiently train the merged LLM.
Organization: leeroo-ai
Home Page: https://www.leeroo.com/
generative-ai llm merge mixture-of-experts nlp fine-tuning large-language-models lora mixture-of-adapters artificial-intelligence

liuqidong07 / moelora-peft
mixture-of-experts,[SIGIR'24] The official implementation code of MOELoRA.
User: liuqidong07
Home Page: https://arxiv.org/abs/2310.18339
chatglm large-language-models low-rank-adaptation mixture-of-experts multi-task multitask-learning parameter-efficient-fine-tuning peft peft-fine-tuning-llm

lucidrains / mixture-of-attention
mixture-of-experts,Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
User: lucidrains
artificial-intelligence attention-mechanisms deep-learning mixture-of-experts routed-attention

lucidrains / mixture-of-experts
mixture-of-experts,A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models
User: lucidrains
artificial-intelligence deep-learning transformer mixture-of-experts

lucidrains / soft-moe-pytorch
mixture-of-experts,Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch
User: lucidrains
artificial-intelligence deep-learning mixture-of-experts transformers

lucidrains / st-moe-pytorch
mixture-of-experts,Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
User: lucidrains
artificial-intelligence deep-learning mixture-of-experts conditional-computation

luodian / generalizable-mixture-of-experts
mixture-of-experts,GMoE could be the next backbone model for many kinds of generalization task.
User: luodian
deep-learning domain-generalization pytorch pytorch-implementation mixture-of-experts

microsoft / deepspeed
mixture-of-experts,DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Organization: microsoft
Home Page: https://www.deepspeed.ai/
deep-learning pytorch gpu machine-learning billion-parameters data-parallelism model-parallelism inference pipeline-parallelism compression

microsoft / tutel
mixture-of-experts,Tutel MoE: An Optimized Mixture-of-Experts Implementation
Organization: microsoft
pytorch moe mixture-of-experts nlp transformer

mryab / learning-at-home
mixture-of-experts,"Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts" (NeurIPS 2020), original PyTorch implementation
User: mryab
Home Page: https://learning-at-home.github.io
pytorch deep-learning distributed-systems mixture-of-experts

pjlab-sys4nlp / llama-moe
mixture-of-experts,⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
Organization: pjlab-sys4nlp
llama llm mixture-of-experts moe continual-pre-training expert-partition

pku-yuangroup / moe-llava
mixture-of-experts,Mixture-of-Experts for Large Vision-Language Models
Organization: pku-yuangroup
Home Page: https://arxiv.org/abs/2401.15947
large-vision-language-model mixture-of-experts moe multi-modal

relf / egobox
mixture-of-experts,Efficient global optimization toolbox in Rust: bayesian optimization, mixture of gaussian processes, sampling methods
User: relf
Home Page: https://joss.theoj.org/papers/10.21105/joss.04737
gaussian-processes global-optimization latin-hypercube-sampling mixture-of-experts surrogate-models

royalskye / routing-mvmoe
mixture-of-experts,[ICML 2024] "MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts"
User: royalskye
Home Page: https://openreview.net/forum?id=lsQnneYa8p
combinatorial-optimization mixture-of-experts vehicle-routing-problem foundation-models generalization learning-to-optimize neural-combinatorial-optimization zero-shot-learning

smtorg / smt
mixture-of-experts,Surrogate Modeling Toolbox
Organization: smtorg
Home Page: https://smt.readthedocs.io/en/stable
surrogate-models derivative sampling mixture-of-experts multi-fidelity predictive-modeling machine-learning

torchmoe / moe-infinity
mixture-of-experts,PyTorch library for cost-effective, fast and easy serving of MoE models.
Organization: torchmoe
inference-engine large-language-models mixture-of-experts huggingface pytorch

tsc2017 / mix-gan
mixture-of-experts,Some recent state-of-the-art generative models in ONE notebook: (MIX-)?(GAN|WGAN|BigGAN|MHingeGAN|AMGAN|StyleGAN|StyleGAN2)(\+ADA|\+CR|\+EMA|\+GP|\+R1|\+SA|\+SN)*
User: tsc2017
Home Page: https://arxiv.org/abs/2007.06418
gan generative-adversarial-network generative-model mixture-model ensemble-learning mixture-of-experts adaptive-discriminator-augmentation mix-gan

umbertocappellazzo / petl_ast
mixture-of-experts,This is the official repository of the papers "Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers" and "Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of Adapters".
User: umbertocappellazzo
Home Page: https://arxiv.org/abs/2312.03694
adapter audio-processing audio-spectrogram-transformer lora parameter-efficient-fine-tuning prompt-tuning transfer-learning mixture-of-experts mixture-of-adapters

unites-lab / mc-smoe
mixture-of-experts,[ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
Organization: unites-lab
Home Page: https://arxiv.org/abs/2310.01334
efficiency merging mixture-of-experts

vita-group / neural-implicit-dict
mixture-of-experts,[ICML 2022] "Neural Implicit Dictionary via Mixture-of-Expert Training" by Peihao Wang, Zhiwen Fan, Tianlong Chen, Zhangyang Wang
Organization: vita-group
dictionary-learning mixture-of-experts neural-implicit-representations

vivamoto / classifier
mixture-of-experts,Machine learning code, derivatives calculation and optimization algorithms developed during the Machine Learning course at Universidade de Sao Paulo. All codes in Python, NumPy and Matplotlib with example in the end of file.
User: vivamoto
python3 machine-learning-algorithms numpy ml mlp cnn ensemble mixture-of-experts optimization logistic-regression

xrsrke / pipegoose
mixture-of-experts,Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
User: xrsrke
megatron megatron-lm transformers 3d-parallelism data-parallelism pipeline-parallelism tensor-parallelism model-parallelism zero-1 large-scale-language-modeling

yangling0818 / realcompo
mixture-of-experts,RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models
User: yangling0818
Home Page: https://arxiv.org/abs/2402.12908
diffusion-models layout-to-image mixture-of-experts text-to-image-diffusion

yeonwoosung / pytorch_mixture-of-experts
mixture-of-experts,PyTorch implementation of moe, which stands for mixture of experts
User: yeonwoosung
mixture-of-experts moe

ymcui / chinese-mixtral
mixture-of-experts,中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）
User: ymcui
Home Page: https://arxiv.org/abs/2403.01851
32k large-language-models llm mixtral mixture-of-experts moe nlp 64k

yuejiangliu / csl
mixture-of-experts,[Preprint] Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts
User: yuejiangliu
alignment mixture-of-experts weak-to-strong