Name: AllenDou
Type: User
Company: Alibaba
Bio: High Performance Gateway(L7) for Alipay, SSL offload, Intel QAT/Cavium Nitrox tech, Altera/Xilinx FPGA, AliRedis author, K8S/Kubeflow, AI Beginner since 2024.
Location: Beijing, China
AllenDou's Projects
AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
scripts help chinese netizen, who uses vpn to combat censorship, by modifying the route table so as routing only the censored ip to the vpn
Script to perform directory diffs using an external diff tool in Git
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
(deal with K8s cluster certificate expired) K8s 集群证书过期处理,更新 kubeadm 生成的证书有效期为 10 年。1.15.x 以上版本可直接 kubeadm alpha certs renew <cert_name> 更新
A high-throughput and memory-efficient inference and serving engine for LLMs