nirantk/agentai

Text to Python Objects via a LLM Function Call

agentai's Issues

Add Azure Form Recognizer Support

In addition to unstructured, have Azure Form Recognizer as an optional dependency. That has great pdf extraction and something I'd like to use more often

Chunking

1. Add reliable chunking for docs (and if easy, tables)
2. Conversation history trimming

See this gist for sample implementations
Reference for how they are used: https://colab.research.google.com/drive/14M4F4FP4NNmSMLOK8aOxfm3MYgj5vNv5?usp=sharing

#6 depends on this

Default to infinite ` max_history_tokens` to be backward compatible.

We should default to infinite max_history_tokens to be backward compatible.

Originally posted by @NirantK in #8 (comment)

Add multiple docs format support

Use: https://github.com/Unstructured-IO/unstructured

For reference, look at how SQLite Utils are organised. These are optional and someone not using them — never needs to care about them

Desired Workflow: Someone points to a pdf file → we read it → chunk it with context → convert to a JSON/Python object

Problem Motivation: Albus

Remember to mark this as an optional dependency in pyproject.toml

Check and confirm if the GPT4 tokeniser is same as gpt2? From what I recall, this is wrong. The tokeniser depends on the LLM.

Check and confirm if the GPT4 tokeniser is same as gpt2? From what I recall, this is wrong. The tokeniser depends on the LLM.

Originally posted by @NirantK in #8 (comment)

No module named 'agentai.openai_function'

Hey, Nirant,

I ran the following as pr the README.md file:
from agentai.api import chat_complete, chat_complete_execute_fn
from agentai.openai_function import tool, ToolRegistry
from agentai.conversation import Conversation
from enum import Enum
weather_registry = ToolRegistry()

Looks like openai_function is not found. This could be because in the updated agentai folder, there is no module named openai_function. Please look into that. Thanks

I believe the openai_function has been moved to annotations from which tool and tool registry can be imported as per the google colab notebook.