BUG: Log in with authentication enabled not working

[AndiMajore]

BUG: Log in with authentication enabled not working

[ChengjieLi28]

Hi, @AndiMajore . I debug your situation one step by one step. I found that If you replace your algorithm config with HS256, then everything works.
It seems that your generated secret key does not match the algorithm RS256. Could you please tell us that what's your OS and how did you generate the secret key?

[ChengjieLi28]

Hi, @AndiMajore . I have updated our document. The core reason is that the secret_key is not generated in the same way if different algorithms need to be used.

[AndiMajore]

Hey, @ChengjieLi28. Thank you very much for that response. I will make sure to generate the key properly again and let you know about the result!

[AndiMajore]

Hey, @ChengjieLi28. Generating a new key in the same way but changing to HS256 helped making the login from commandline and for the Python client work, or at least not break. When I now try to launch a model directly after i logged in it works for Python and also authentication through the web UI. Through commandline, it does give me an error still:

(base) root@df6d69022608:/opt/conda/lib/python3.9# xinference login -e http://localhost:9997 --username $ADMIN_USER --password $ADMIN_PASSWORD
(base) root@df6d69022608:/opt/conda/lib/python3.9# xinference launch --model-name llama-2-chat --size-in-billions 13 --model-format pytorch --quantization none  --n-gpu 1
Traceback (most recent call last):
  File "/opt/conda/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 622, in model_launch
    client._set_token(get_stored_token(endpoint, client))
  File "/opt/conda/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 83, in get_stored_token
    raise RuntimeError("Cannot find access token, please login first!")
RuntimeError: Cannot find access token, please login first!

[ChengjieLi28]

Hey, @ChengjieLi28. Generating a new key in the same way but changing to HS256 helped making the login from commandline and for the Python client work, or at least not break. When I now try to launch a model directly after i logged in it works for Python and also authentication through the web UI. Through commandline, it does give me an error still:

(base) root@df6d69022608:/opt/conda/lib/python3.9# xinference login -e http://localhost:9997 --username $ADMIN_USER --password $ADMIN_PASSWORD
(base) root@df6d69022608:/opt/conda/lib/python3.9# xinference launch --model-name llama-2-chat --size-in-billions 13 --model-format pytorch --quantization none  --n-gpu 1
Traceback (most recent call last):
  File "/opt/conda/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 622, in model_launch
    client._set_token(get_stored_token(endpoint, client))
  File "/opt/conda/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 83, in get_stored_token
    raise RuntimeError("Cannot find access token, please login first!")
RuntimeError: Cannot find access token, please login first!

Please try to launch model using this command:

xinference launch -e http://localhost:9997 ...

In the cli, xinference stores the needed credentials(token) locally, distinguished by endpoint. For now, we do not recognize localhost as 127.0.0.1. If you do not specify -e option, xinference uses http://127.0.0.1:9997 by default.In the future, we can improve this.

[AndiMajore]

Perfect, works now!
Thank you!