QUESTION: `xinference registrations` command error: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) about inference HOT 9 OPEN

BaiMeiyingxue commented on June 23, 2024

QUESTION: `xinference registrations` command error: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

from inference.

Comments (9)

BaiMeiyingxue commented on June 23, 2024

the version of package is

----------------------------- ------------
accelerate                    0.28.0
addict                        2.4.0
aiobotocore                   2.7.0
aiofiles                      23.2.1
aiohttp                       3.9.3
aioitertools                  0.11.0
aioprometheus                 23.12.0
aiosignal                     1.3.1
aliyun-python-sdk-core        2.15.0
aliyun-python-sdk-kms         2.16.2
altair                        5.2.0
annotated-types               0.6.0
anyio                         4.3.0
async-timeout                 4.0.3
attrs                         23.2.0
auto_gptq                     0.7.1
bcrypt                        4.1.2
bitsandbytes                  0.43.0
botocore                      1.31.64
certifi                       2024.2.2
cffi                          1.16.0
charset-normalizer            3.3.2
chatglm-cpp                   0.3.1
click                         8.1.7
cloudpickle                   3.0.0
colorama                      0.4.6
coloredlogs                   15.0.1
contourpy                     1.2.0
controlnet-aux                0.0.7
crcmod                        1.7
cryptography                  42.0.5
cycler                        0.12.1
datasets                      2.18.0
diffusers                     0.26.3
dill                          0.3.8
diskcache                     5.6.3
distro                        1.9.0
ecdsa                         0.18.0
einops                        0.7.0
exceptiongroup                1.2.0
fastapi                       0.110.0
ffmpy                         0.3.2
filelock                      3.13.1
fonttools                     4.49.0
frozenlist                    1.4.1
fsspec                        2023.10.0
gast                          0.5.4
gekko                         1.0.7
gradio                        4.21.0
gradio_client                 0.12.0
h11                           0.14.0
httpcore                      1.0.4
httptools                     0.6.1
httpx                         0.27.0
huggingface-hub               0.21.4
humanfriendly                 10.0
idna                          3.6
imageio                       2.34.0
importlib_metadata            7.0.2
importlib_resources           6.3.0
Jinja2                        3.1.3
jmespath                      0.10.0
joblib                        1.3.2
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
kiwisolver                    1.4.5
lazy_loader                   0.3
llama_cpp_python              0.2.56
markdown-it-py                3.0.0
MarkupSafe                    2.1.5
matplotlib                    3.8.3
mdurl                         0.1.2
modelscope                    1.13.1
mpmath                        1.3.0
msgpack                       1.0.8
multidict                     6.0.5
multiprocess                  0.70.16
networkx                      3.2.1
ninja                         1.11.1.1
numpy                         1.26.4
nvidia-cublas-cu12            12.1.3.1
nvidia-cuda-cupti-cu12        12.1.105
nvidia-cuda-nvrtc-cu12        12.1.105
nvidia-cuda-runtime-cu12      12.1.105
nvidia-cudnn-cu12             8.9.2.26
nvidia-cufft-cu12             11.0.2.54
nvidia-curand-cu12            10.3.2.106
nvidia-cusolver-cu12          11.4.5.107
nvidia-cusparse-cu12          12.1.0.106
nvidia-nccl-cu12              2.18.1
nvidia-nvjitlink-cu12         12.4.99
nvidia-nvtx-cu12              12.1.105
openai                        1.13.3
opencv-python                 4.9.0.80
optimum                       1.17.1
orjson                        3.9.15
oss2                          2.18.4
packaging                     24.0
pandas                        2.2.1
passlib                       1.7.4
peft                          0.9.0
pillow                        10.2.0
pip                           23.3.1
platformdirs                  4.2.0
protobuf                      4.25.3
psutil                        5.9.8
pyarrow                       15.0.1
pyarrow-hotfix                0.6
pyasn1                        0.5.1
pycparser                     2.21
pycryptodome                  3.20.0
pydantic                      2.6.4
pydantic_core                 2.16.3
pydub                         0.25.1
Pygments                      2.17.2
pynvml                        11.5.0
pyparsing                     3.1.2
python-dateutil               2.9.0.post0
python-dotenv                 1.0.1
python-jose                   3.3.0
python-multipart              0.0.9
pytz                          2024.1
PyYAML                        6.0.1
quantile-python               1.1
ray                           2.9.3
referencing                   0.33.0
regex                         2023.12.25
requests                      2.31.0
rich                          13.7.1
rouge                         1.0.1
rpds-py                       0.18.0
rsa                           4.9
ruff                          0.3.2
s3fs                          2023.10.0
safetensors                   0.4.2
scikit-image                  0.22.0
scikit-learn                  1.4.1.post1
scipy                         1.12.0
semantic-version              2.10.0
sentence-transformers         2.5.1
sentencepiece                 0.2.0
setuptools                    68.2.2
shellingham                   1.5.4
simplejson                    3.19.2
six                           1.16.0
sniffio                       1.3.1
sortedcontainers              2.4.0
sse-starlette                 2.0.0
starlette                     0.36.3
sympy                         1.12
tabulate                      0.9.0
tblib                         3.0.0
threadpoolctl                 3.3.0
tifffile                      2024.2.12
tiktoken                      0.6.0
timm                          0.9.16
tokenizers                    0.15.2
tomli                         2.0.1
tomlkit                       0.12.0
toolz                         0.12.1
torch                         2.1.2
torchvision                   0.16.2
tqdm                          4.66.2
transformers                  4.38.2
transformers-stream-generator 0.0.5
triton                        2.1.0
typer                         0.9.0
typing_extensions             4.10.0
tzdata                        2024.1
urllib3                       2.0.7
uvicorn                       0.28.0
uvloop                        0.19.0
vllm                          0.3.0
watchfiles                    0.21.0
websockets                    11.0.3
wheel                         0.41.2
wrapt                         1.16.0
xformers                      0.0.23.post1
xinference                    0.9.2
xoscar                        0.2.1
xxhash                        3.4.1
yapf                          0.40.2
yarl                          1.9.4
zipp                          3.18.0```

from inference.

ChengjieLi28 commented on June 23, 2024

the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred

  File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations
    client = RESTfulClient(base_url=endpoint)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__
    self._check_cluster_authenticated()
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated
    f"Failed to get cluster information, detail: {response.json()['detail']}"
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```

Does the xinference start normally on port 9997?

from inference.

BaiMeiyingxue commented on June 23, 2024

the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred

  File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations
    client = RESTfulClient(base_url=endpoint)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__
    self._check_cluster_authenticated()
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated
    f"Failed to get cluster information, detail: {response.json()['detail']}"
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```

Does the xinference start normally on port 9997?

2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997```
everything seems ok

from inference.

ChengjieLi28 commented on June 23, 2024

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine.
Could you please paste your command that is used for starting xinference?

from inference.

BaiMeiyingxue commented on June 23, 2024

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

from inference.

ChengjieLi28 commented on June 23, 2024

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

Have you tried other commands? Do all commands run with this problem?

from inference.

BaiMeiyingxue commented on June 23, 2024

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

Have you tried other commands? Do all commands run with this pr

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

Have you tried other commands? Do all commands run with this problem?

yes, I tried to register a model with commands, and the seem error occured

from inference.

YoungerXu commented on June 23, 2024

If you have enabled a proxy, please try disabling it and then attempt the operation again. This maybe resolve the issue you encountered.

from inference.

dongrixinyu commented on June 23, 2024

have encountered the same bug.

from inference.

QUESTION: `xinference registrations` command error: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) about inference HOT 9 OPEN

Comments (9)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent