Comments (1)
There is indeed an issue. These two model files have been divided into two parts. We need to figure out how to resolve this problem.
from inference.
Related Issues (20)
- 增大 context length 原本能够运行的长度的对话也会触发 CUDA OOM
- Considering let the user specify the model path( from local machine ) or download repo( custom repo in hf or modelscope) for build-in model HOT 3
- 建议增加界面n_ctx参数配置。Qwen1.5-14B-Chat-GGUF模型默认小显存无法运行 HOT 3
- ENH: Support vllm params for UI HOT 2
- 支持Lora模型加载
- BLD: Official CPU docker image
- BUG: Failed to launch qwe1.5-14b-GGUF using Xinferrence but works fine in pure llama_cpp python HOT 3
- BUG: Error when destroy generator in xoscar HOT 2
- BUG: qwen1.5 gptq int8 errored HOT 2
- FEAT: Add a command / SDK interface to query which models are able to run on VLLM
- BUG请问我在安装pip install "xinference[all]"时出错 HOT 6
- 有docker部署embedding的示例吗 HOT 1
- Server error: 500 - [address=127.0.0.1:50182, pid=36312] [WinError None] 客户端没有所需的特权 HOT 1
- BUG: for windows cannot start with IP 0.0.0.0 HOT 21
- DOC: add how to contribute doc HOT 1
- 显存占用双倍异常 HOT 7
- max_tokens设置似乎出现异常
- BUG: Qwen1.5-chat 0.5B cannot be downloaded from modelscope
- QUESTION: 以xinference作为后台服务,不能同时多人进行提问 HOT 3
- 想问下如何对qwen1.5进行多机多卡批量推理
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from inference.