shahzebali42/llm-poc

This repo is a collection of various PoCs (Proof-of-Concepts) to interface custom data using LLMs.

Sarvadnya (सर्वज्ञ), an All-Knowing Micro SaaS!!

Chatbots can be real WoW!! The recent evidence is: ChatGPT. Now that they are more human-like with the latest LLMs (Large Language Models). But these LLMs are Pretrained on their own (HUGE) data. Mere mortals don't have any ways ($$, time, expertise) to train own LLMs. Some do have facility to get fine-tuned on custom corpus, but limited. Custom fine tuning of text documents is being provided by many.

This repo is a collection of various PoCs (Proof-of-Concepts) to interface custom data using LLMs.

Stretch (RnD) goals:

• {KG + LLM} chatbot Building LLM based Chatbot on Knowledge Graph
• Knowledge as a Service (KaaS) Building Knowledge Graph from Text and serving it as a Service
• LLM models for Indic (especially Sanskrit) languages. Here is collection of similar efforts going on Awesome AI By Bharat (AABB)

Pathways

• Enterprise: Google Cloud: Gen AI, Doc AI, Vertex AI: Skills Boost paths, Professional ML Certification
• Open Source: Langchain, HuggingFace, Streamlit: Custom fine-tuned models

Why LangChain based Implementations ?:

• Local (secure), no over-the-net API/web calls
• Open source, Free via HuggingFace
• Python!! end-to-end, with Streamlit as UI
• Huge support, community, opportunities

Publications so far

• SaaS LLM
• AskAlmanackBot
• GST FAQs Bot

References

• LangChain How to and guides
• Building the Future with LLMs, LangChain, & Pinecone
• LangChain for Gen AI and LLMs - James Briggs
• Finetuning GPT-3 David Shapiro ~ AI
• Build overpowered AI apps with the OP stack (OpenAI + Pinecone)
• Learn about AI Language Models and Reinforcement Learning Kamalraj M M
• GPT-4 & LangChain Tutorial: How to Chat With A 56-Page PDF Document (w/Pinecone)
• LangChain - Data Independent
• Node Classification on Knowledge Graphs using PyTorch Geometric
• Geometric Deep Learning
• PyTorch Geometric
• Machine and Language Learning Lab IISc
• Semantic Web India Enables organizations generate value from Data using AI, Knowledge Discovery
• Cambridge Semantics
• Kenome Partha Talukdar. Helping enterprises make sense of dark data using cutting-edge Machine Learning, NLP, and Knowledge Graphs.
• Knowledge graphs
• [Geometric Deep Learning)[https://geometricdeeplearning.com/]
• Learning on Graphs Conference
• Group Equivariant Deep Learning (UvA - 2022)

Bottom-line

• Not looking for Success, but Wonder!!
• तमसो मा ज्योतिर्गमय : From Dark (hidden in text data) to Light (insights)