e-johnstonn/briefgpt

Locally hosted tool that connects documents to LLMs for summarization and querying, with a simple GUI.

briefgpt's Issues

Inference too slow

Hi!

Thank you for sharing your good. Its really a great help to understand langchain and LLMs.
I am running the model using NVIDIA V100 32GB ram. The model takes a lot of time to response to the questions. The size pdf document I am loading is around 1.4 MB.

Can you please comment on how to improve the response time?

Regards

run local_app.py LlamaCpp load error

models/ggml-vic13b-q5_1.bin
llama.cpp: loading model from models/ggml-vic13b-q5_1.bin
error loading model: unknown (magic, version) combination: 67676a74, 00000002; is this really a GGML file?
llama_init_from_file: failed to load model

Open API key

Testing

example in test.env doesn't match if statement

In test.env, it mentions MODEL_TYPE = LlamaCpp or GPT4ALL

However, in local_app.py the if statement is as follows:

elif model_type == 'GPT4All':

This causes it to never match, as the L's are uppercase in test.env, but lowercase in the if statement.

I tried to create a branch and submit a PR for this but got 401, though that could be entirely my fault

Many eBooks are in EPUB format。I hope this project can support it

rt

bug at chat_utils.py line 99

chat_utils.py line 99
When the llm is None, the "output = llm(message)" will throw error.

Local Summarizer Support

Hey! Is there any way to get the summarizer working off a local LMM?

The only way I've found to use the summarizer is with an OpenAI key with gpt3.5 or gpt4.

ModuleNotFoundError: No module named 'sklearn'

This happens when running streamlit run main.py

I am using conda for a virtual env and installed the dependencies in the requirements file. I had to manually remove the pywin32 dependency since I'm on an m1 MBP and that doesn't appear to exist.

Please advise.

Support for alternative OpenAI's endpoints

It'd be a good idea to support other endpoints of OpenAI services (like Microsoft Azure). Presumably the env file would need other variables like OPENAI_API_BASE.

Repo is missing license

The repo is missing a license, which makes it technically a closed sourced, even though I believe it is intended to be open source. I suggest adding an MIT license

3. Set your API key in test.env - How to create API Key

Can you please help with how to create API key using test.env

add support for folder(multiple files) chat for the local run

Hello,
I think it would be great and handy if multiple files could be searched just like main.py for local_app.py. Since similar code/feature already exist in main.py, it could be quickly added. Thank you for the project.