run local_app.py LlamaCpp load error

[HardSoft2023]

run local_app.py LlamaCpp load error

[e-johnstonn]

my guess is there's some issue between the version of llama.cpp and the model you're trying to use. the llama.cpp version I used was 0.1.50 - are you using the same version?

if so, maybe try other versions of llama.cpp - pip install llama-cpp-python===version number - try with 0.1.45 maybe

[HardSoft2023]

I use 0.1.49.

Actually， the problem is that the file name I specified is incorrect.
It should be models/ggml-vic13b-q5_0.bin。
so,I suggest replacing the link address in readme from https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/blob/main/ggml-vic13b-q5_1.bin to https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/blob/main/ggml-old-vic13b-q5_0.bin

[e-johnstonn]

sorry i dont quite understand, is it working for you now? are you saying the link in the readme is to the wrong version of llama

[HardSoft2023]

yes ,It works, now.
when I change 'models/ggml-vic13b-q5_1.bin' --->to 'models/ggml-vic13b-q5_0.bin'

[HardSoft2023]

Let me use pictures to explain my problems clearly.

1. This URL in README Llama SHA256 is OK

2. But can not load properly

3. Use the new URL works good

all this . I made a PR.

[krstrid]

Downgrade to 0.1.48 if you want to use LlamaCpp with gpt4all. Got it to work using HardSofts instructions as well as AndreaKiro's.

zylon-ai/private-gpt#220