mindinventory/mindsql

MindSQL: A Python Text-to-SQL RAG Library simplifying database interactions. Seamlessly integrates with PostgreSQL, MySQL, SQLite, Snowflake, and BigQuery. Powered by GPT-4 and Llama 2, it enables natural language queries. Supports ChromaDB and Faiss for context-aware responses.

🧠 MindSQL

MindSQL is a Python RAG (Retrieval-Augmented Generation) Library designed to streamline the interaction between users and their databases using just a few lines of code. With seamless integration for renowned databases such as PostgreSQL, MySQL, and SQLite, MindSQL also extends its capabilities to major databases like Snowflake and BigQuery by extending the IDatabase Interface. This library utilizes large language models (LLM) like GPT-4, Llama 2, Google Gemini, and supports knowledge bases like ChromaDB and Faiss.

🚀 Installation

To install MindSQL, you can use pip:

pip install mindsql

MindSQL requires Python 3.10 or higher.

💡 Usage

# !pip install mindsql

from mindsql.core import MindSQLCore
from mindsql.databases import Sqlite
from mindsql.llms import GoogleGenAi
from mindsql.vectorstores import ChromaDB

# Add Your Configurations
config = {"api_key": "YOUR-API-KEY"}

# Choose the Vector Store. LLM and DB You Want to Work With And
# Create MindSQLCore Instance With Configured Llm, Vectorstore, And Database
minds = MindSQLCore(
    llm=GoogleGenAi(config=config),
    vectorstore=ChromaDB(),
    database=Sqlite()
)

# Create a Database Connection Using The Specified URL
connection = minds.database.create_connection(url="YOUR_DATABASE_CONNECTION_URL")

# Index All Data Definition Language (DDL) Statements in The Specified Database Into The Vectorstore
minds.index_all_ddls(connection=connection, db_name='NAME_OF_THE_DB')

# Index Question-Sql Pair in Bulk From the Specified Example Path
minds.index(bulk=True, path="your-qsn-sql-example.json")

# Ask a Question to The Database And Visualize The Result
response = minds.ask_db(
    question="YOUR_QUESTION",
    connection=connection,
    visualize=True
)

# Extract And Display The Chart From The Response
chart = response["chart"]
chart.show()

# Close The Connection to Your DB
connection.close()

📁 Code Structure

• _utils: Utility modules containing constants and a logger.
• _helper: The helper module.
• core: The main core module, minds_core.py.
• databases: Database-related modules.
• llms: Modules related to Language Models.
• testing: Testing scripts.
• vectorstores: Modules related to vector stores.
• poetry.lock and pyproject.toml: Poetry dependencies and configuration files.
• tests: Testcases.

🤝 Contributing Guidelines

Thank you for considering contributing to our project! Please follow these guidelines for smooth collaboration:

1. Fork the repository and create your branch from master.

2. Ensure your code adheres to our coding standards and conventions.

3. Test your changes thoroughly and add a test case in the tests folder.

4. Submit a pull request with a clear description of the problem and solution.

   Learn more

🐛 Bug Reports

If you encounter a bug while using MindSQL, help us resolve it by following these steps:

1. Check existing issues to see if the bug has been reported.

2. If not, open a new issue with a detailed description, including steps to reproduce and relevant screenshots or error messages.

   Learn more

🚀 Feature Requests

We welcome suggestions for new features or improvements to MindSQL. Here's how you can request a new feature:

1. Check existing feature requests to avoid duplication.

2. If your feature request is unique, open a new issue and describe the feature you would like to see.

3. Provide as much context and detail as possible to help us understand your request.

   Learn more

📣 Feedback

We value your feedback and strive to improve MindSQL. Here's how you can share your thoughts with us:

• Open an issue to provide general feedback, suggestions, or comments.
• Be constructive and specific in your feedback to help us understand your perspective better.

Thank you for your interest in contributing to our project! We appreciate your support and look forward to working with you. 🚀

🌟 Contributors

GitHub Profile	Link + Image	Name
siddhant-mi		Siddhant Pandey
ishika-mi		Ishika Shah
Hasmukhsuthar05		Hasmukh Suthar
krishna-thakkar-mi		Krishna Thakkar
UjjawalKRoy		Ujjawal Roy

mindsql's Issues

minds.ask_db() : list index out of range

I am trying to run a local Llama 2 model against my SQL database on ubuntu. However, I'm encountering the following range error.

WARNING — 2024-03-07 03:27:38,715 — Minds Core — Function:ask_db — Line:498 — An unexpected error occurred: list index out of range
Res: {'error': 'list index out of range'}

Here is the code:

from mindsql.core.mindsql_core import MindSQLCore
from mindsql.llms import LlamaCpp
from mindsql.databases import MySql
from mindsql.vectorstores import Faiss

config = {
    "model_path": "/home/guitmonk/Downloads/llama-2-7b.Q3_K_L.gguf"
}

class MindSqlGenAI(Faiss, LlamaCpp, MySql):
    def __init__(self, config=config):
        Faiss.__init__(self, config=config)
        LlamaCpp.__init__(self, config=config)
        MySql.__init__(self, config=config)

minds = MindSqlGenAI(config=config)
connection = minds.create_connection(<SQL_URL>)

# Index DB Schmas
ddls = minds.get_all_ddls(connection=connection, database="test")

for ind in ddls.index:
    minds.index_ddl(ddls["DDL"][ind])

# Provide Example query-SQL pairs
minds.index(bulk=False, path="examples.json")

# ask questions
res = minds.ask_db(question="How many total clients are there?", connection=connection)
print("Res: ", res)

connection.close()

I've tried to debug it but I'm not sure where is this error originating from.

Adding custom configurations for Llama

The context window in Llama is set to 512 by default. This can cause the prompt size to exceed the context window size if the DB schema is large enough.

It would be great if custom configuration for the Llama function from LlamaCpp can be passed in the config object for better control.

How does this differ from Vanna?

Hey there! This seems eerily similar to Vanna:

https://github.com/vanna-ai/vanna

I’m curious why our projects need to be different

Support for wikisql or open surce llms

Hi, good work. Are there plans to add support for any open source llms that does not require api keys?