BUG

[vikrantrathore]

BUG

[vikrantrathore]

Ok after doing a second try by removing the vllm from the git it started to work now but UI error is not elimited it still throws an error:

Xinference ui is not built at expected directory: /home/projects/insan_inference/.venv/lib/python3.11/site-packages/xinference/web/ui/build/
            To resolve this warning, navigate to /home/projects/insan_inference/.venv/lib/python3.11/site-packages/xinference/web/ui/
            And build the Xinference ui by running "npm run build"

  warnings.warn(

[ChengjieLi28]

@vikrantrathore Could you paste out your command that you pip install from git?

[vikrantrathore]

The error is through poetry which uses pip behind. The command is poetry add git+https://github.com/xorbitsai/inference within a local project.

Once the installation fails at installing xinference, I can go activate the venv by poetry shell and then install using pip install git+https://github.com/xorbitsai/inference. It works, but then there is an error of Web UI which is using npm and need to run

npm install
npm run build

This will build the UI. Also vllm used is 0.4.3 and wanted to try 0.5.0 from the git given it supports gemma-2 9B and 27B but couldn't do it due to other dependencies.

Also sglang support gemma-it but when I use sglang worker for gemma-it model it throws an error model not supported so was trying to install it from git directly to see if can use gemma-it with sglang.

[vikrantrathore]

It seems the error still persist when I try to install it from local directory like poetry add -e ../inference/.["all"], probably need to test more extensively with poetry.