zhangdy-6483 Goto Github PK

computer-architecture

Python Learning of D.Y. Zhang

cutlass

CUDA Templates for Linear Algebra Subroutines

fastertransformer

Transformer related optimization, including BERT, GPT

flash-attention

Fast and memory-efficient exact attention

google-research

Google Research

megatron-lm

Ongoing research training transformer language models at scale, including: BERT & GPT-2

nccl

Optimized primitives for collective multi-GPU communication

paddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

paddlefl

Federated Deep Learning in PaddlePaddle

paddlefleetx

Paddle Distributed Training Examples. 飞桨分布式训练示例 Resnet Bert GPT MOE DataParallel ModelParallel PipelineParallel HybridParallel AutoParallel Zero Sharding Recompute GradientMerge Offload AMP DGC LocalSGD Wide&Deep

paddlenlp

👑 Easy-to-use and powerful NLP library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis and 🖼 Diffusion AIGC system etc.

ppl.llm.kernel.cuda

pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

tensorrt-llm

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

timeloop

triton

Development repository for the Triton language and compiler

verilog

Verilog study

xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.