This repository contains a Google Colab notebook that demonstrates setting up a retrieval-based QA system to answer questions about Business
The notebook does the following:
- Loads the Wikipedia Simple Text Embedding dataset from Pinecone and preprocesses it
- Indexes the document embeddings into a Pinecone vector database
- Defines an OpenAI text embedding model to embed question queries
- Creates a Pinecone vectorstore to interface the index and embeddings
- Defines a LangChain retrieval QA chain using ChatGPT, with the Pinecone vectorstore as the retriever
- Answers an example insurance question using the retrieval QA chain
The notebook requires the following key libraries:
- LangChain v0.0.162
- OpenAI v0.27.7
- TikToken v0.4.0
- Pinecone Python SDK v2.2.1
- Pinecone Datasets v0.5.0rc10
To install requirements:
!pip install -qU \ langchain==0.0.162 \ openai==0.27.7 \ tiktoken==0.4.0 \ "pinecone-client[grpc]"==2.2.1 \ pinecone_datasets=='0.5.0rc10'
You will also need API keys for:
- Pinecone
- OpenAI
To run the notebook:
- Clone this repo
- Open the notebook in Google Colab
- Run all cells - you will need to configure Pinecone and OpenAI keys
- Modify the example question and rerun as desired
The key components:
RetrievalQA
- LangChain QA chain- Pinecone - Vector database
- OpenAI Embeddings - Text embedding model
The main ways to customize this:
- Try different embedding models
- Change the dataset indexed into Pinecone
- Modify the query questions
- Experiment with other LLMs than ChatGPT
This demo uses: