modeltc Goto Github PK

aaai2023_eampd

AAAI2023 Efficient and Accurate Models towards Practical Deep Learning Baseline

awesome-lm-system

Summary of system papers/frameworks/codes/tools on training or serving large model

chatglm-6b

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

deepspeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

dipoorlet

Offline Quantization Tools for Deploy.

easyllm

Built upon Megatron-Deepspeed and HuggingFace Trainer, EasyLLM has reorganized the code logic with a focus on usability. While enhancing usability, it also ensures training efficiency.

fcpts

general-sam

A general suffix automaton implementation in Rust with Python bindings

general-sam-py

Python bindings for general-sam and some utilities

greedy-tokenizer

Greedily tokenize strings with the longest tokens iteratively.

imagenet-s

Robustness for real-world system noise

l2_compression

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

llmc

This is the official implementation of "LLM-QBench: A Benchmark Towards the Best Practice for Post-training Quantization of Large Language Models", and it is also an efficient LLM compression tool with various advanced compression methods, supporting multiple inference backends.

lpcv2021_winner_solution

lpcv_2023_solution

mqbench

Model Quantization Benchmark

mqbench-paper

msbench

A tool for model sparse based on torch.fx

mtc-token-healing

Token healing implementation in Rust

nart

NART = NART is not A RunTime, a deep learning inference framework.

nnlqp

outlier_suppression_plus

Official implementation of the EMNLP23 paper: Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

prototype

pyrotom

Python Code Hotfix and Refactor on the fly

pyvlova

Yet another Polyhedra Compiler for DeepLearning

qllm

[ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models"

rank_dataset

PyTorch Dataset Rank Dataset

statecs

systemnoise_web