Ollama asking for API Key: "Unable to validate API key".

[mjtechguy]

Ollama asking for API Key: "Unable to validate API key".

[HyperUpscale]

Wait....
IT WORKS ... WHEN I REMOVE THE STREAM OPTION FROM THE MODEL

[mjtechguy]

Would it be possible to call the ollama embedding endpoint?

https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings

This would allow for the offloading to a remote ollama node running on a GPU.

Thanks!

[HyperUpscale]

I don't understand ... there are always new error from somewhere @_@

In the docker console:

Error: Request to Ollama server failed: 404 Not Found
at OllamaEmbeddings._request (/app/node_modules/@langchain/community/dist/embeddings/ollama.cjs:101:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async RetryOperation._fn (/app/node_modules/p-retry/index.js:50:12) {
attemptNumber: 7,
retriesLeft: 0
}<

It seems to me that the web interface is kinda independent from the config, or the docker.
Because now the Web UI accept the model and embedding settings, but the dont work on the backend (docker).
Example of what I am saying: on the Web page in the config the address ONLY "http://host.docker.internal:11434" is accepted, but it that doesn't work when the docker image tries to get my ollama (I forgot to mention it is on WSL, not docker)

So all in all it seems like the local host IP network is conflicting the code. It does it with LiteLLM the same way Dialoqbase.

Somehow you (also LiteLLM) are reffering to the local host in specific way, because of tens of other projects work with the current setup.
One of them is Ollama web UI - using docker and it finds and works with ollama on "http://host.docker.internal:11434"

Another tip: My host IPs are a complicated story... not sure if from the WSL of from the Docker... For that is what confuses the dialoqbase.

Now I am trying to figure out the docker network... ... ...

[HyperUpscale]

IDK

[HyperUpscale]

ollama version is 0.1.20

[n4ze3m]

Hey @mjtechguy, I have released the bug fixes that will resolve your issues. :)

[n4ze3m]

Hey, the error is because of the ollama embedding you selected, which is deprecated. You can add Ollama embedding models the same way you added the Ollama chat model by going to 'Models' and selecting 'Embedding'

[HyperUpscale]

Hey, the error is because of the ollama embedding you selected, which is deprecated. You can add Ollama embedding models the same way you added the Ollama chat model by going to 'Models' and selecting 'Embedding'

I decoded your reply:

And it seem to pass that problem.

But then we have a NEW message :)

That means it doesn't update the config...

I tried manually, but I don't know how it will work:

Gives the same NEW error.

PLEASE HELP!

BTW it doesn't work with localhost, 127.0.0.1... it finds the ollama server with host.docker.internal. Maybe that is related IDK

[n4ze3m]

Gives the same NEW error.

Hey, sorry about the error. What error is it throwing now?

[n4ze3m]

In the docker console:

Interesting that you added the Ollama chat model via the editor, but are you still experiencing the issue, right?

If you successfully added the Ollama model, then there is no problem with the chat model. The issue is caused by the embedding model you configured. Can you double-check the model ID?

[n4ze3m]

Hey, I think I found the issue not related to Docker (just tested it on my Windows WSL) too.

Can you try using zephyr:7b-beta as the chat model along with any other embedding model (use Xenova/all-Min-L6-v2) and send a message? It should work.

There will be a new release in 2 or 3 hours that fixes the above issues (hopefully).

Also, is http://host.docker.internal:11434 working perfectly

[HyperUpscale]

http://host.docker.internal:11434/ is the only way that HALF works for me... I don't know if that will be for everyone :D

[n4ze3m]

That's a strange bug. I will look into it. Which version of Ollama are you using?

Also, do not remove {question} from the prompt.

[n4ze3m]

I tested Ollama mainly on Llama2 and Mistral; both of them work fine for me. I released a few Ollama-related fixes; you can pull the latest in 1 or 2 hours.

[n4ze3m]

Try another model you have locally to see if it's working fine. :)

[n4ze3m]

Yes, You can configure Ollama embedding like chat models by following these steps:

• Go to admin models.
• Select 'embedding.'
• Add the Ollama URL and choose the model from which you want to generate embedding. (The UI will be enhanced in the coming version.)

Please let me know if you encounter any issues.