Topic: llm-inference Goto Github
Some thing interesting about llm-inference
Some thing interesting about llm-inference
llm-inference,Run any Large Language Model behind a unified API
User: 1b5d
llm-inference,irresponsible innovation. Try now at https://chat.dev/
Organization: anarchy-ai
Home Page: https://anarchy.ai/
llm-inference,Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.
User: b4rtaz
llm-inference,Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.
Organization: bentoml
Home Page: https://bentoml.com
llm-inference,Bespoke Automata is a GUI and deployment pipline for making complex AI agents locally and offline
User: c0demunk33
llm-inference,🪶 Lightweight OpenAI drop-in replacement for Kubernetes
User: chenhunghan
llm-inference,Code examples and resources for DBRX, a large language model developed by Databricks
Organization: databricks
Home Page: https://www.databricks.com/
llm-inference,📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
User: deftruth
Home Page: https://github.com/DefTruth/Awesome-LLM-Inference
llm-inference,LLM.swift is a simple, and readable library which lets you locally interact with LLMs with ease for macOS, iOS, visionOS, watchOS, and tvOS.
User: eastriverlee
llm-inference,Fast Inference of MoE Models with CPU-GPU Orchestration
Organization: efeslab
Home Page: https://arxiv.org/abs/2402.07033
llm-inference, Embedding Studio is a framework which allows you transform your Vector Database into a feature-rich Search Engine.
Organization: eulersearch
Home Page: https://embeddingstud.io/
llm-inference,Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
Organization: fasterdecoding
Home Page: https://sites.google.com/view/medusa-llm
llm-inference,The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.
Organization: flagai-open
llm-inference,FlashInfer: Kernel Library for LLM Serving
Organization: flashinfer-ai
Home Page: https://flashinfer.ai
llm-inference,LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.
User: ghimiresunil
llm-inference,Efficient AI Inference & Serving
Organization: hpcaitech
Home Page: https://hpc-ai.com/
llm-inference,Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
User: inferflow
llm-inference,⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
Organization: intel
llm-inference,An innovative library for efficient LLM inference via low-bit quantization
Organization: intel
Home Page: https://github.com/intel/neural-speed
llm-inference,LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Organization: internlm
Home Page: https://lmdeploy.readthedocs.io/en/latest/
llm-inference,LLMs and Machine Learning done easily
Organization: kenza-ai
Home Page: https://kenza-ai.github.io/sagify/
llm-inference,LLMs as Copilots for Theorem Proving in Lean
Organization: lean-dojo
Home Page: https://leandojo.org
llm-inference,Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.
Organization: lightning-ai
Home Page: https://lightning.ai
llm-inference,本项目旨在分享大模型相关技术原理以及实战经验。
User: liguodongiot
Home Page: https://www.zhihu.com/column/c_1456193767213043713
llm-inference,Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
User: liltom-eth
llm-inference,AICI: Prompts as (Wasm) Programs
Organization: microsoft
llm-inference,A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
Organization: microsoft
Home Page: https://microsoft.github.io/autogen/
llm-inference,Reference implementation of Mistral AI 7B v0.1 model.
Organization: mistralai
Home Page: https://mistral.ai/
llm-inference,Morpheus - A Network For Powering Smart Agents - Compute + Code + Capital + Community
Organization: morpheusais
Home Page: https://mor.org/
llm-inference,AI-powered cybersecurity chatbot designed to provide helpful and accurate answers to your cybersecurity-related queries and also do code analysis and scan analysis.
User: morpheuslord
llm-inference,Sparsity-aware deep learning inference runtime for CPUs
Organization: neuralmagic
Home Page: https://neuralmagic.com/deepsparse/
llm-inference,gpt4all: run open-source LLMs anywhere
Organization: nomic-ai
Home Page: https://gpt4all.io
llm-inference,Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
Organization: nvidia
Home Page: https://nvidia.github.io/GenerativeAIExamples/latest/index.html
llm-inference,OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
Organization: openvinotoolkit
Home Page: https://docs.openvino.ai
llm-inference,Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Organization: predibase
Home Page: https://loraexchange.ai
llm-inference,A multi-platform SwiftUI frontend for running local LLMs with Apple's MLX framework.
Organization: preternaturalai
llm-inference,Tune LLM in few lines of code
Organization: promptslab
llm-inference,This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.
Organization: ray-project
llm-inference,RayLLM - LLMs on Ray
Organization: ray-project
Home Page: https://aviary.anyscale.com
llm-inference,A tool for generating function arguments and choosing what function to call with local LLMs
User: rizerphe
Home Page: https://local-llm-function-calling.readthedocs.io/
llm-inference,LLM (Large Language Model) FineTuning
User: rohan-paul
llm-inference,GPU environment and cluster management with LLM support
Organization: run-ai
Home Page: https://www.genv.dev
llm-inference,EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
Organization: safeailab
Home Page: https://arxiv.org/abs/2401.15077
llm-inference,High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
Organization: sjtu-ipads
llm-inference,LLMFlows - Simple, Explicit and Transparent LLM Apps
User: stoyan-stoyanov
Home Page: https://llmflows.readthedocs.io
llm-inference,Finetune LLMs on K8s by using Runbooks
Organization: substratusai
Home Page: https://www.substratus.ai
llm-inference,🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.
Organization: superduperdb
Home Page: https://superduperdb.com
llm-inference,A library to communicate with ChatGPT, Claude, Copilot, Gemini, HuggingChat, and Pi
User: ugorsahin
llm-inference,A high-performance inference system for large language models, designed for production environments.
Organization: vectorch-ai
llm-inference,🦖 Stateful Serverless Framework for building Geo-distributed Edge AI Infra
Organization: yomorun
Home Page: https://yomo.run
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.