safevideo/autollm

Ship RAG based LLM web apps in seconds.

autollm's Issues

How to load PDF and other Docs

How to load PDF and other documents, here are just told about .md

Is conversation context included in the query?

The conversation context is not included in the search does not work

HuggingFace Embeddings Problems

After creates all embeddings get a "currently loading" from embeddings model

Traceback (most recent call last):
  File "/home/frankie/Documents/falopitaConIA/app.py", line 53, in <module>
    query_engine = AutoQueryEngine.from_defaults(
  File "/home/frankie/.local/lib/python3.10/site-packages/autollm/auto/query_engine.py", line 258, in from_defaults
...
...
litellm.exceptions.APIConnectionError: Model BAAI/bge-small-zh is currently loading

here is my code:

# import required functions, classes
from autollm import AutoQueryEngine, AutoFastAPI
from autollm.utils.document_reading import read_github_repo_as_documents, read_files_as_documents
import os
import uvicorn

required_exts = [".md"]    # optional, only read files with these extensions
relative_folder_path = "docs"   # relative path from the repo root to the folder containing documents

documents = read_files_as_documents(input_dir="docs", required_exts=required_exts)

# llm params
llm_model = "ollama/llama2"
#llm_max_tokens = 512
llm_temperature = 1
llm_api_base = "http://localhost:11434"

os.environ['HUGGINGFACE_API_KEY'] = "hf_XXXXXXXXXXXXXXXvHu"

system_prompt = """
"""
query_wrapper_prompt = '''
The document information is below.
---------------------
{context_str}
---------------------
Using the document information and mostly relying on it,
answer the query.
{query_str}
'''
enable_cost_calculator = True
embed_model="huggingface/BAAI/bge-small-zh"
chunk_size = 248
chunk_overlap = 48
context_window = 8200

# vector store params
vector_store_type = "LanceDBVectorStore"
lancedb_uri = "./.lancedb"
lancedb_table_name = "vectors"
use_async = True
exist_ok = True
overwrite_existing = True

# query engine params
similarity_top_k = 3
response_mode = "compact_accumulate"
structured_answer_filtering= False

query_engine = AutoQueryEngine.from_defaults(
    documents=documents,
    llm_model=llm_model,
    llm_temperature=llm_temperature,
    llm_api_base=llm_api_base,
    system_prompt=system_prompt,
    query_wrapper_prompt=query_wrapper_prompt,
    enable_cost_calculator=enable_cost_calculator,
    embed_model=embed_model, 
    chunk_size=chunk_size,
    chunk_overlap=chunk_overlap,
    context_window=context_window,
    vector_store_type=vector_store_type,
    lancedb_uri=lancedb_uri,
    lancedb_table_name=lancedb_table_name,
    use_async=use_async,
    exist_ok=exist_ok,
    overwrite_existing=overwrite_existing,
    similarity_top_k=similarity_top_k,
    response_mode=response_mode,
    structured_answer_filtering=structured_answer_filtering,
)
#response = query_engine.query(query_input)
#print(response.response)

app = AutoFastAPI.from_query_engine(query_engine)
uvicorn.run(app, host="0.0.0.0", port=8000)

请问支持华为昇腾910b吗

Explain the Parameters of AutoQueryEngine

Quickstart has undefined objects leading error

I came across this project on LinkedIn that attracted my attetion. Then I decided to clone it to explore its functionality. Regrettably, I encountered issues with the quick start instructions provided in the Readme.md file.

query_engine = AutoQueryEngine.from_parameters(documents: List[llama_index.Documents]) 
# documents and llama_index objects are not defined, even List is not defined.

I would greatly appreciate it if these instructions could be rectified or alternatively supplemented with a more straightforward example. Thank you.

How to use locally hosted LLM's ?

Hi

The code snippet provided in more llms: section of Readme, fails with this error
raise OpenAIError( openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

from autollm import AutoQueryEngine
from autollm.utils.document_reading import read_files_as_documents

llm_model = "ollama/llama2"
llm_api_base = "http://localhost:11434"

documents = read_files_as_documents(input_dir="./data", required_exts=[".md"])

AutoQueryEngine.from_defaults(
     documents=documents,
     llm_model=llm_model,
     llm_api_base=llm_api_base,
)

I assume using OpenAI models is not mandatory

Questions in Description:

1. How to see the sources of answers?
2. How to add metadata or anyway to include metadata?

llm builder demo

llm-builder-demo.mp4

Api does not maintain saperate sessions and uses same context and chat history for different sessions

I tried using the fastapi server and found out that when I call api from two different hosts , the query engine considers call coming from a single source and uses the existing context to answer the prompt,
Ideally the query engine should be aware of different clients and maintain different contexts per client

Issue with pip install autollm[readers]

When I try pip install autollm[readers], I'm getting the following warning (I'm using pip 23.0.1, but the latest version is 24.0):

DEPRECATION: docx2txt is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at pypa/pip#8559

I really like the autollm functionality. If this issue can be fixed, I'd really appreciate it.

Improving Tables Extraction and Retrieval!

Except for tables everything is working fine!
How to retrieve tables accurately.

Can it run on CPU

Can the setup run on CPUs too?

RuntimeError: asyncio.run() cannot be called from a running event loop

Notebook: https://colab.research.google.com/github/safevideo/autollm/blob/main/examples/quickstart.ipynb

import error autollm

Hi Team,

I tried to install autollm and imported it in venv (python3.8). I am getting below error message. Please check with this.

from autollm import AutoQueryEngine

class AgentAction(Serializable): File "pydantic/main.py", line 198, in pydantic.main.ModelMetaclass.__new__ File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__ File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare File "pydantic/fields.py", line 668, in pydantic.fields.ModelField._type_analysis File "/usr/lib/python3.8/typing.py", line 774, in __subclasscheck__ return issubclass(cls, self.__origin__) TypeError: issubclass() arg 1 must be a class

Python: 3.8
NVIDIA-SMI 520.61.05 Driver Version: 520.61.05 CUDA Version: 11.8

GPU 0: NVIDIA GeForce RTX 3090 (UUID: GPU-e2ce5f43-3889-5510-ec0c-831eab8c6ff0)

Please check this.

Thanks in advance.
Santhosh

API Request Parameters and Reusing Vector Store Without Regenerating Embeddings

1. What parameter to send as GET/POST on API?
2. How to use Vector Store without creating a new one, every time. because it always run generate embeddings even though vector store already there?

Local embedding model not working

LLM Provider is not provided. because you have passed "Local"

Processing documents via the API?

Hi, I like the idea of the API and I have played around with the examples for the regular script although there's not much documentation in terms of the API.

This is my config.yaml:

version: '1.0'  # Version of this configuration file
tasks:
  - name: "summarize"
    system_prompt: "You are an expert ai assistant specialized in summarization."  # System prompt for this task
    vector_store_params:
      vector_store_type: "LanceDBVectorStore"
      uri: "tmp/lancedb"
      collection_name: "summarize"
    llm_params:
      model: "gpt-3.5-turbo"
    service_context_params:
      chunk_size: 1024
    query_engine_params:
      similarity_top_k: 5
    enable_cost_calculator: true
  - name: "qa"
    system_prompt: "You are a friendly ai assistant specialized in question answering."  # System prompt for this task
    vector_store_params:
      vector_store_type: "LanceDBVectorStore"
      uri: "tmp/lancedb"
      collection_name: "qa"
    llm_params:
      model: "gpt-4"
    service_context_params:
      chunk_size: 1024
    query_engine_params:
      similarity_top_k: 3
    enable_cost_calculator: false

And this is my api.py:

import uvicorn

from autollm import AutoFastAPI

app = AutoFastAPI.from_config("config.yaml")

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

How can I load a GitHub Readme.md from a URL via this API? From the generated OpenAPI documentation itself, I can see there seem to be 3 available query params:

{
  "task": "summarize",
  "user_query": "Summarize the Readme.MD file from the provided Github Repo...,
  "streaming": false
}

Is it possible to somehow pass the Readme URL somehow? When I try to pass it within the user_query itself, it gives an error like: "I'm sorry, but I cannot directly access external websites or specific GitHub repositories. However, if you provide me with the content of the README.md file from the GitHub repository, I can help you summarize it.".

Thanks in advance

Deprecation Warnings

I ran the following code successfully:

from autollm import AutoQueryEngine, read_files_as_documents

documents = read_files_as_documents(input_dir="data")
query_engine = AutoQueryEngine.from_defaults(documents)

response = query_engine.query("Summarize this essay.")
print(response)

However, I'm getting the following warnings (mostly deprecation warnings):

`LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.chat_models import ChatAnyscale.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\vpera\OneDrive\Documents\GitHub\OncologyClinicalTrial.env\lib\site-packages\langchain\chat_models_init_.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.chat_models import ChatOpenAI.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\vpera\OneDrive\Documents\GitHub\OncologyClinicalTrial.env\lib\site-packages\langchain\embeddings_init_.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.embeddings import HuggingFaceBgeEmbeddings.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\vpera\OneDrive\Documents\GitHub\OncologyClinicalTrial.env\lib\site-packages\langchain\embeddings_init_.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.embeddings import HuggingFaceEmbeddings.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\vpera\OneDrive\Documents\GitHub\OncologyClinicalTrial.env\lib\site-packages\langchain\llms_init_.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import AI21.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\vpera\OneDrive\Documents\GitHub\OncologyClinicalTrial.env\lib\site-packages\langchain\llms_init_.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import Cohere.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\vpera\OneDrive\Documents\GitHub\OncologyClinicalTrial.env\lib\site-packages\langchain\llms_init_.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import FakeListLLM.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\vpera\OneDrive\Documents\GitHub\OncologyClinicalTrial.env\lib\site-packages\langchain\llms_init_.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import OpenAI.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
2024-02-14 11:13:45,111 - autollm - WARNING - use_async is only supported for SimpleVectorStore. Ignoring use_async.`

Unable to install autollm using pip

` pip install autollm
Collecting autollm
Using cached autollm-0.1.4-py3-none-any.whl.metadata (14 kB)
INFO: pip is looking at multiple versions of autollm to determine which version is compatible with other requirements. This could take a while.
Using cached autollm-0.1.3-py3-none-any.whl.metadata (14 kB)
Using cached autollm-0.1.2-py3-none-any.whl.metadata (14 kB)
Using cached autollm-0.1.1-py3-none-any.whl.metadata (14 kB)
Using cached autollm-0.0.22-py3-none-any.whl.metadata (14 kB)
Using cached autollm-0.0.21-py3-none-any.whl.metadata (14 kB)
Using cached autollm-0.0.20-py3-none-any.whl.metadata (14 kB)
Using cached autollm-0.0.19-py3-none-any.whl.metadata (13 kB)
INFO: pip is still looking at multiple versions of autollm to determine which version is compatible with other requirements. This could take a while.
Using cached autollm-0.0.18-py3-none-any.whl.metadata (13 kB)
Using cached autollm-0.0.17-py3-none-any.whl.metadata (12 kB)
Using cached autollm-0.0.16-py3-none-any.whl.metadata (12 kB)
Using cached autollm-0.0.15-py3-none-any.whl.metadata (12 kB)
Using cached autollm-0.0.14-py3-none-any.whl.metadata (12 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Using cached autollm-0.0.13-py3-none-any.whl.metadata (10 kB)
Using cached autollm-0.0.12-py3-none-any.whl.metadata (10 kB)
Using cached autollm-0.0.11-py3-none-any.whl.metadata (9.6 kB)
Using cached autollm-0.0.10-py3-none-any.whl.metadata (9.6 kB)
Using cached autollm-0.0.9-py3-none-any.whl.metadata (9.0 kB)
Using cached autollm-0.0.8-py3-none-any.whl.metadata (9.0 kB)
Using cached autollm-0.0.7-py3-none-any.whl.metadata (9.0 kB)
Using cached autollm-0.0.6-py3-none-any.whl.metadata (9.0 kB)
Using cached autollm-0.0.5-py3-none-any.whl.metadata (10 kB)
Using cached autollm-0.0.4-py3-none-any.whl.metadata (10 kB)
Using cached autollm-0.0.3-py3-none-any.whl.metadata (10 kB)
Using cached autollm-0.0.2-py3-none-any.whl.metadata (10 kB)
Using cached autollm-0.0.1-py3-none-any.whl.metadata (10 kB)
ERROR: Cannot install autollm==0.0.1, autollm==0.0.10, autollm==0.0.11, autollm==0.0.12, autollm==0.0.13, autollm==0.0.14, autollm==0.0.15, autollm==0.0.16, autollm==0.0.17, autollm==0.0.18, autollm==0.0.19, autollm==0.0.2, autollm==0.0.20, autollm==0.0.21, autollm==0.0.22, autollm==0.0.3, autollm==0.0.4, autollm==0.0.5, autollm==0.0.6, autollm==0.0.7, autollm==0.0.8, autollm==0.0.9, autollm==0.1.1, autollm==0.1.2, autollm==0.1.3 and autollm==0.1.4 because these package versions have conflicting dependencies.

The conflict is caused by:
autollm 0.1.4 depends on llama-index==0.9.10
autollm 0.1.3 depends on llama-index==0.9.10
autollm 0.1.2 depends on llama-index==0.9.4
autollm 0.1.1 depends on llama-index==0.9.4
autollm 0.0.22 depends on llama-index==0.8.58
autollm 0.0.21 depends on llama-index==0.8.58
autollm 0.0.20 depends on llama-index==0.8.58
autollm 0.0.19 depends on llama-index==0.8.58
autollm 0.0.18 depends on llama-index==0.8.58
autollm 0.0.17 depends on llama-index==0.8.58
autollm 0.0.16 depends on llama-index==0.8.54
autollm 0.0.15 depends on llama-index==0.8.54
autollm 0.0.14 depends on llama-index==0.8.54
autollm 0.0.13 depends on llama-index==0.8.54
autollm 0.0.12 depends on llama-index==0.8.54
autollm 0.0.11 depends on llama-index==0.8.50
autollm 0.0.10 depends on llama-index==0.8.50
autollm 0.0.9 depends on llama-index==0.8.50
autollm 0.0.8 depends on llama-index==0.8.46
autollm 0.0.7 depends on llama-index==0.8.46
autollm 0.0.6 depends on llama-index==0.8.46
autollm 0.0.5 depends on llama-index==0.8.46
autollm 0.0.4 depends on llama-index==0.8.46
autollm 0.0.3 depends on llama-index==0.8.46
autollm 0.0.2 depends on llama-index==0.8.46
autollm 0.0.1 depends on llama-index==0.8.46

To fix this you could try to:

1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts`

and when i try to install llama-index given version it says version not found

pip install llama-index==0.9.10
ERROR: Ignored the following yanked versions: 0.4.4, 0.4.4.post1
ERROR: Ignored the following versions that require a different python version: 0.8.43 Requires-Python >=3.8.1,<3.12; 0.8.43.post1 Requires-Python >=3.8.1,<3.12; 0.8.44 Requires-Python >=3.8.1,<3.12; 0.8.45 Requires-Python >=3.8.1,<3.12; 0.8.45.post1 Requires-Python >=3.8.1,<3.12; 0.8.46 Requires-Python >=3.8.1,<3.12; 0.8.47 Requires-Python >=3.8.1,<3.12; 0.8.48 Requires-Python >=3.8.1,<3.12; 0.8.49 Requires-Python >=3.8.1,<3.12; 0.8.50 Requires-Python >=3.8.1,<3.12; 0.8.51 Requires-Python >=3.8.1,<3.12; 0.8.51.post1 Requires-Python >=3.8.1,<3.12; 0.8.52 Requires-Python >=3.8.1,<3.12; 0.8.53 Requires-Python >=3.8.1,<3.12; 0.8.53.post3 Requires-Python >=3.8.1,<3.12; 0.8.54 Requires-Python >=3.8.1,<3.12; 0.8.55 Requires-Python >=3.8.1,<3.12; 0.8.56 Requires-Python >=3.8.1,<3.12; 0.8.57 Requires-Python >=3.8.1,<3.12; 0.8.58 Requires-Python >=3.8.1,<3.12; 0.8.59 Requires-Python >=3.8.1,<3.12; 0.8.61 Requires-Python >=3.8.1,<3.12; 0.8.62 Requires-Python >=3.8.1,<3.12; 0.8.63.post1 Requires-Python >=3.8.1,<3.12; 0.8.63.post2 Requires-Python >=3.8.1,<3.12; 0.8.64 Requires-Python >=3.8.1,<3.12; 0.8.64.post1 Requires-Python >=3.8.1,<3.12; 0.8.65 Requires-Python >=3.8.1,<3.12; 0.8.66 Requires-Python >=3.8.1,<3.12; 0.8.67 Requires-Python >=3.8.1,<3.12; 0.8.68 Requires-Python >=3.8.1,<3.12; 0.8.69 Requires-Python >=3.8.1,<3.12; 0.8.69.post1 Requires-Python >=3.8.1,<3.12; 0.8.69.post2 Requires-Python >=3.8.1,<3.12; 0.9.0 Requires-Python >=3.8.1,<3.12; 0.9.0.post1 Requires-Python >=3.8.1,<3.12; 0.9.0a1 Requires-Python >=3.8.1,<3.12; 0.9.0a2 Requires-Python >=3.8.1,<3.12; 0.9.0a3 Requires-Python >=3.8.1,<3.12; 0.9.1 Requires-Python >=3.8.1,<3.12; 0.9.10 Requires-Python >=3.8.1,<3.12; 0.9.10a1 Requires-Python >=3.8.1,<3.12; 0.9.10a2 Requires-Python >=3.8.1,<3.12; 0.9.11 Requires-Python >=3.8.1,<3.12; 0.9.11.post1 Requires-Python >=3.8.1,<3.12; 0.9.2 Requires-Python >=3.8.1,<3.12; 0.9.3 Requires-Python >=3.8.1,<3.12; 0.9.3.post1 Requires-Python >=3.8.1,<3.12; 0.9.4 Requires-Python >=3.8.1,<3.12; 0.9.5 Requires-Python >=3.8.1,<3.12; 0.9.6 Requires-Python >=3.8.1,<3.12; 0.9.6.post1 Requires-Python >=3.8.1,<3.12; 0.9.6.post2 Requires-Python >=3.8.1,<3.12; 0.9.7 Requires-Python >=3.8.1,<3.12; 0.9.8 Requires-Python >=3.8.1,<3.12; 0.9.8.post1 Requires-Python >=3.8.1,<3.12; 0.9.9 Requires-Python >=3.8.1,<3.12
ERROR: Could not find a version that satisfies the requirement llama-index==0.9.10 (from versions: 0.4.4.post2, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14, 0.4.15, 0.4.16, 0.4.17, 0.4.18, 0.4.19, 0.4.20, 0.4.21, 0.4.22, 0.4.22.post1, 0.4.23, 0.4.24, 0.4.25, 0.4.26, 0.4.27, 0.4.28, 0.4.29, 0.4.30, 0.4.31, 0.4.32, 0.4.33, 0.4.34, 0.4.35, 0.4.35.post1, 0.4.36, 0.4.37, 0.4.38, 0.4.39, 0.4.40, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.5.9, 0.5.10, 0.5.11, 0.5.12, 0.5.13, 0.5.13.post1, 0.5.15, 0.5.16, 0.5.17, 0.5.17.post1, 0.5.18, 0.5.19, 0.5.20, 0.5.21, 0.5.22, 0.5.23, 0.5.23.post1, 0.5.25, 0.5.26, 0.5.27, 0.6.0a1, 0.6.0a2, 0.6.0a3, 0.6.0a4, 0.6.0a5, 0.6.0a6, 0.6.0a7, 0.6.0, 0.6.1, 0.6.2, 0.6.4, 0.6.5, 0.6.6, 0.6.7, 0.6.8, 0.6.9, 0.6.10, 0.6.10.post1, 0.6.11, 0.6.12, 0.6.13, 0.6.14, 0.6.15, 0.6.16, 0.6.16.post1, 0.6.17, 0.6.18, 0.6.19, 0.6.20, 0.6.21.post1, 0.6.22, 0.6.23, 0.6.24, 0.6.25, 0.6.25.post1, 0.6.26, 0.6.27, 0.6.28, 0.6.29, 0.6.30, 0.6.31, 0.6.32, 0.6.33, 0.6.34, 0.6.34.post1, 0.6.35, 0.6.36, 0.6.37, 0.6.38, 0.6.38.post1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.7.7, 0.7.8, 0.7.9, 0.7.10, 0.7.10.post1, 0.7.11, 0.7.11.post1, 0.7.12, 0.7.13, 0.7.14, 0.7.15, 0.7.16, 0.7.17, 0.7.18, 0.7.19, 0.7.20, 0.7.21, 0.7.22, 0.7.23, 0.7.24.post1, 0.8.0, 0.8.1, 0.8.1.post1, 0.8.2, 0.8.2.post1, 0.8.3, 0.8.4, 0.8.5, 0.8.5.post1, 0.8.5.post2, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.8.10, 0.8.10.post1, 0.8.11, 0.8.11.post1, 0.8.11.post2, 0.8.11.post3, 0.8.12, 0.8.13, 0.8.14, 0.8.15, 0.8.16, 0.8.17, 0.8.18, 0.8.19, 0.8.20, 0.8.21, 0.8.22, 0.8.23, 0.8.23.post1, 0.8.24, 0.8.24.post1, 0.8.25, 0.8.26, 0.8.26.post1, 0.8.27, 0.8.28a1, 0.8.28, 0.8.29, 0.8.29.post1, 0.8.30, 0.8.31, 0.8.32, 0.8.33, 0.8.34, 0.8.35, 0.8.36, 0.8.37, 0.8.38, 0.8.39, 0.8.39.post2, 0.8.40, 0.8.41, 0.8.42, 0.9.12a1, 0.9.12a2, 0.9.12a3, 0.9.12a4, 0.9.12a5, 0.9.12, 0.9.13, 0.9.14, 0.9.14.post1, 0.9.14.post2, 0.9.14.post3, 0.9.15, 0.9.15.post1, 0.9.15.post2, 0.9.16, 0.9.16.post1)
ERROR: No matching distribution found for llama-index==0.9.10

error with very large chunk_size

Error if you insert a lot of text, it is better to read the text in the input window and not skip such a request

Traceback (most recent call last):
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\queueing.py", line 427, in call_prediction
output = await route_utils.call_process_api(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\route_utils.py", line 232, in call_process_api
output = await app.get_blocks().process_api(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1497, in process_api
result = await self.call_function(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1119, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 665, in wrapper
response = f(*args, **kwargs)
File "d:\Programm\AI\autollm\app.py", line 54, in greet
return query_engine.query(query).response
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\query\base.py", line 32, in query
return self._query(str_or_query_bundle)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\query_engine\retriever_query_engine.py", line 182, in _query
response = self._response_synthesizer.synthesize(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\response_synthesizers\base.py", line 148, in synthesize
response_str = self.get_response(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\response_synthesizers\compact_and_refine.py", line 37, in get_response
new_texts = self._make_compact_text_chunks(query_str, text_chunks)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\response_synthesizers\compact_and_refine.py", line 52, in _make_compact_text_chunks
return self._service_context.prompt_helper.repack(max_prompt, text_chunks)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 223, in repack
text_splitter = self.get_text_splitter_given_prompt(prompt, padding=padding)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 186, in get_text_splitter_given_prompt
chunk_size = self._get_available_chunk_size(prompt, num_chunks, padding=padding)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 171, in _get_available_chunk_size
available_context_size = self._get_available_context_size(prompt)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 152, in _get_available_context_size
raise ValueError(
ValueError: Calculated available context size -4574 was not non-negative.

upload header

default lancedb error

it happens at python3.9, macosx13.5.2 , with pip install autollm, FastAPI example code ,after run it, it throw errors at following:

AutoQueryEngine.from_defaults(
  File "/Users/yangboz/anaconda3/envs/py39/lib/python3.9/site-packages/autollm/auto/query_engine.py", line 258, in from_defaults
    return create_query_engine(
  File "/Users/yangboz/anaconda3/envs/py39/lib/python3.9/site-packages/autollm/auto/query_engine.py", line 105, in create_query_engine
    vector_store_index = AutoVectorStoreIndex.from_defaults(
  File "/Users/yangboz/anaconda3/envs/py39/lib/python3.9/site-packages/autollm/auto/vector_store_index.py", line 84, in from_defaults
    lancedb_uri = AutoVectorStoreIndex._validate_and_setup_lancedb_uri(
  File "/Users/yangboz/anaconda3/envs/py39/lib/python3.9/site-packages/autollm/auto/vector_store_index.py", line 143, in _validate_and_setup_lancedb_uri
    raise ValueError(
ValueError: No existing database found at ./.lancedb. Please provide a valid lancedb uri.
Sentry is attempting to send 2 pending events
Waiting up to 2 seconds
Press Ctrl-C to quit

any ideas? thanks.

add docs - Working with local model or local api

Please add docs.

Example only using free Hugging Face inference and embeddings

Pls. create a working example which only uses free Hugging Face endpoints for both embeddings and completion.
I tried to tweak existing example using
AutoQueryEngine.from_config()
by the docs but it still uses OpenAI's endpoints, so I was wrong.
Probably others also would prefer this, without spending any money for OpenAI API :)

document loading error

documents = read_files_as_documents(input_dir="examples/data" , recursive=True)

PS D:\Programm\AI\autollm> & C:/Users/Aleksandr/AppData/Local/Programs/Python/Python310/python.exe d:/Programm/AI/autollm/app.py
INFO:autollm.utils.document_reading:Found 1 'documents'.
Parsing documents into nodes: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 69.86it/s]
Generating embeddings: 0%| | 0/4 [00:00<?, ?it/s]Embedding Token Usage: 13738
Generating embeddings: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:01<00:00, 2.50it/s]
Traceback (most recent call last):
File "d:\Programm\AI\autollm\app.py", line 35, in
query_engine = AutoQueryEngine.from_parameters(
File "d:\Programm\AI\autollm\autollm\auto\query_engine.py", line 121, in from_parameters
return create_query_engine(
File "d:\Programm\AI\autollm\autollm\auto\query_engine.py", line 43, in create_query_engine
vector_store_index = AutoVectorStoreIndex.from_defaults(
File "d:\Programm\AI\autollm\autollm\auto\vector_store_index.py", line 89, in from_defaults
index = VectorStoreIndex.from_documents(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\base.py", line 102, in from_documents
return cls(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\vector_store\base.py", line 49, in init
super().init(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\base.py", line 71, in init
index_struct = self.build_index_from_nodes(nodes)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\vector_store\base.py", line 238, in build_index_from_nodes
return self._build_index_from_nodes(nodes)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\vector_store\base.py", line 226, in _build_index_from_nodes
self._add_nodes_to_index(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\vector_store\base.py", line 187, in _add_nodes_to_index
new_ids = self._vector_store.add(nodes)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\vector_stores\lancedb.py", line 122, in add
tbl.add(data)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\lancedb\table.py", line 575, in add
data = _sanitize_data(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\lancedb\table.py", line 69, in _sanitize_data
data = _sanitize_schema(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\lancedb\table.py", line 981, in _sanitize_schema
[data[name] for name in schema.names], schema=schema
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\lancedb\table.py", line 981, in
[data[name] for name in schema.names], schema=schema
File "pyarrow\table.pxi", line 1538, in pyarrow.lib._Tabular.getitem
return self.column(key)
File "pyarrow\table.pxi", line 1624, in pyarrow.lib._Tabular.column
return self._column(self._ensure_integer_index(i))
File "pyarrow\table.pxi", line 1560, in pyarrow.lib._Tabular._ensure_integer_index
raise KeyError("Field "{}" does not exist in schema"
KeyError: 'Field "original_file_path" does not exist in schema'
Sentry is attempting to send 2 pending events
Waiting up to 2 seconds
Press Ctrl-Break to quit

image sources

Does autoLLM support HuggingFace LLM's

need to integrating huggingface models also other than just proprietary LLM's.

Custom Embedding Model

Can i use a open Embedding Model instead openAI?

Multi-modal LLM Support

For Image, Audio and Video Embeddings!

Tip: Creating Comprehensive Documentation

Local embedding model always download.

The local embedding model always downloads in the following temp location and then re-download every time rebooted.
Can you fix this and change /tmp to the local persisted directory like ~/.tmp inside user's home?

What is difference between system prompt and query prompt wrapper?

I'm a bit confused!

query_engine issue

code example comes from https://abrahimzaman360.medium.com/introduction-to-autollm-c8cd31be2a5f

after run it ,throws error as following:

query_engine = AutoQueryEngine.from_parameters(documents=documents)
AttributeError: type object 'AutoQueryEngine' has no attribute 'from_parameters'

any ideas? thanks

ModuleNotFoundError: No module named 'bs4'

from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'

please add BeautifulSoup lib to the package.

There is an example of just a Python script

There is an example of just a Python script, I don’t understand how I can make it work from another provider

"openrouter/openai/gpt-3.5-turbo": {
    "max_tokens": 4095,
    "input_cost_per_token": 0.0000015,
    "output_cost_per_token": 0.000002,
    "litellm_provider": "openrouter",
    "mode": "chat"
},

`import os
import gradio as gr
from autollm import AutoQueryEngine
from autollm.utils.document_reading import read_github_repo_as_documents
from pydantic import BaseModel, Field

os.environ["OPENROUTER_API_KEY"] = ...
model = "openrouter/openai/gpt-3.5-turbo"
api_base = "https://openrouter.ai/api/v1"

git_repo_url = "https://github.com/ultralytics/ultralytics.git"
relative_folder_path = "docs" # относительный путь к папке с документами в репозитории
required_exts = [".md"] # расширения файлов для чтения

documents = read_github_repo_as_documents(
git_repo_url=git_repo_url, relative_folder_path=relative_folder_path, required_exts=required_exts
)

llm_params = {
"model": model,
"api_base": api_base,
}

query_engine = AutoQueryEngine.from_parameters(
documents=documents,
system_prompt='ты помогаешь его дорабатывать',
query_wrapper_prompt='о чем этот репо',
enable_cost_calculator=True,
llm_params=llm_params,
vector_store_params={
"vector_store_type": "LanceDBVectorStore",
"uri": "/tmp/lancedb",
"table_name": "lancedb",
"nprobs": 20
},
service_context_params={"chunk_size": 4095},
query_engine_params={"similarity_top_k": 10},
)

query = "How to perform tiled inference with SAHI and YOLOv8, include code snippets"
response = query_engine.query(query)

print(response.response)

def greet(query):
return query_engine.query(query).response

demo = gr.Interface(fn=greet, inputs="text", outputs="text")
demo.launch()`

torch.cuda.OutOfMemoryError: CUDA out of memory.

hi 👋
I refer to the README and issue, discussion to load my local model, but he reports an error.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacty of 23.69 GiB of
which 30.12 MiB is free. Including non-PyTorch memory, this process has 23.54 GiB memory in use. Of the allocated memory
23.29 GiB is allocated by PyTorch, and 1.05 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory 
is large try setting max_split_

But this text-generated model runs perfectly fine on this machine!
This is very strange.
Here is my code.

from autollm import AutoQueryEngine
from autollm.utils.document_reading import read_files_as_documents
import os

os.environ["HUGGINGFACE_API_KEY"] = "hf_xxxx"
documents = read_files_as_documents(input_dir="/home/yang/rag-app/docs")

query_engine = AutoQueryEngine.from_defaults(
     documents=documents,
     embed_model='local:FlagAlpha/Llama2-Chinese-7b-Chat',
     llm_model='FlagAlpha/Llama2-Chinese-7b-Chat',
)

def query(prompt):
  response = query_engine.query(
    prompt
 )
  return response.response

and the example can't work https://github.com/safevideo/autollm#supports-100-llms

TypeError: AutoQueryEngine.from_parameters() got an unexpected keyword argument 'llm_model'

Does autoLLM support local LLMs?

With few exceptions, most supported LLMs are 4K context window. Local LLMs at least give the opportunity for 16K, 32K and more.