adriacabeza/erudito

Erudito: Easy API/CLI to ask questions about your documentation

Hi there 👋

• 🔭 I’m currently working on the Security Monitoring product at @datadog
• 😄 Pronouns: he/him
• 📌 Location: Barcelona

erudito's Issues

Re-build documentation

Documentation that is published on the github pages website is out-of-sync with the actual state of the docs/ folder.

For example the command for running the fastapi server on the website is wrong and it's corrected in the main branch of this repository.

Can we get this to run on OOba 4bit quantized models?

That would be amazing :)

Unable to load lora-quantize-model

I have been trying to test this but the application is unable to load the model throwing out the following error. I am using quantised lora model from gpt4all repo for testing on a wsl distribution on windows 11 with a .txt data in data folder and the model in the model folder. I am using the following command

python -m src.ingest data/ models/gpt4all/gpt4all-lora-quantized.bin

It throws out following error

Model Path= models/gpt4all/gpt4all-lora-quantized.bin
llama.cpp: loading model from models/gpt4all/gpt4all-lora-quantized.bin
error loading model: llama.cpp: tensor '�~5��x�{�d��HuV' should not be 131072-dimensional
llama_init_from_file: failed to load model

Any suggestions?

Great Project - Can't read pdfs though - PdfFileReader is deprecated and was removed in PyPDF2 3.0.0. Use PdfReader instead.

Excellent project, thanks for making this for us folks who struggle to put these things together.

When I try to get it to read my files, I get error - PdfFileReader is deprecated and was removed in PyPDF2 3.0.0. Use PdfReader instead.

'Llama' object has no attribute 'n_embd'

Running python -m src.ingest hit an error on ingest.py:

39: embeddings = np.empty((len(chunks), llama.n_embd()))

AttributeError: 'Llama' object has no attribute 'n_embd'

I'm not sure why, because this shows the definition of n_embd. However, updating to llama-cpp-python==0.2.11 in requirements.lock resolved the problem.