Comments (2)
qinxuye
commented on June 27, 2024
1
I don't know if OpenVINO can work on intel NPU, we are introducing OpenVINO in #1677 .
from inference.
yw662
commented on June 27, 2024
Yeah Intel officially uses openVINO in the demo. It may not work very well with too large models though.
from inference.
Related Issues (20)
- qwen2工具调用，对话中输出了部分思考过程 HOT 9
- 运行速度很慢，为什么？ HOT 3
- 支持将第三方llm平台转为openai-like接口嘛，比如火山方舟、百度千帆 HOT 1
- BUG xinference、glm4工具调用报错400
- v0.12.2没有上传到dockerhub HOT 6
- [QUESTION] can't install "xinference[all]", No module named 'Cython' HOT 2
- [QUESTION] Failed to import module 'SentenceTransformer' HOT 1
- docker部署时：Unexpected error from cudaGetDeviceCount() HOT 6
- xinference 格式初始化llm，sse 流式输出有问题 HOT 1
- CURL没返回值 HOT 3
- xinference 使用 glm4-9b-chat function calling 报错
- A bug about the web ui and the llm model.
- xinference升级到0.12.2后突然的OOM HOT 1
- 启动xinference后新注册了本地模型，但是在重启xinference后，注册的模型没有记录需要重新注册 After starting xinference, a new local model was registered, but after restarting xinference, the registered model has no record and needs to be re-registered. HOT 3
- ValueError: [address=127.0.0.1:37657, pid=3125985] User-specified max_model_len (4096) is greater than the derived max_model_len (seq_length=2048 or model_max_length=None in model's config.json). This may lead to incorrect model outputs or CUDA errors. Make sure the value is correct and within the model context size.
- embedding model bge-m3 how to set return_sparse=True ? HOT 2
- BUG Unable to load CogVlm2 Model
- 请教大家，客户端数量暴增，并且一直在累计，是什么导致的呢，如何设置最大限制
- docker实例无法启动运行 HOT 2
- BUG 多个请求qwen2-7b模型时，推理会报错 probability tensor contains either `inf`, `nan` or element < 0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from inference.