manufy/ai-intro

Artificial Intelligence LLM experiments and quick intros using Ollama, GPT4All, OpenAI and HuggingFace models with LangChain and DeepLake vector store

ai-intro

This repo contains small projects using current AI frameworks

Updated to May 2024

This frameworks, tools and models are used:

• Langchain framework
• DeepLake vector datastore
• Local Ollama/Orca/Phi3 models
• HuggingFace LLM, sentiment and speech recognition models

Ensure to have about 50Gb free for HuggingFace and Ollama model downloads

Environment install

• Create and activate conda environment (optional)

  environment-and-requirements/create-conda-env.sh environment-and-requirements/activate-conda-env.sh

• Install ActiveLoop dependencies

  environment-and-requirements/install-activeloop-requirements.sh

• Instal HuggingFace dependencies

  environment-and-requirements/install-huggingface-requirements.sh

• Create a .env file with API keys and credentials

  To be documented later using load_dotenv module

## Deprecation warnings

The most annoying thing is that the activeloop course is using deprecated modules. The code will use the updated modules, but still some things needs to be changed to remove all deprecation warnings. This depend on third party updates, despite the samples are using latest modules, if a module depends on another module, this can throw a warning because the included module was still not updated at the time of writing this repo There are comments on the code to ignore the warnings to have a nice clean output.

Courses

• ActiveLoop LangChain & Vector Databases in Production

Platforms

• HuggingFace Quick Tour

• OpenAI Platform API

• ActiveLoop DataLake

LangChain tutorials

• DataCamp

• Langchain Docs

Models

• OLlama

ai-intro's Issues

add testing in pipeline

currently shows

============================= test session starts ==============================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/runner/work/ai-intro/ai-intro
collected 0 items

============================ no tests ran in 0.02s =============================

temporary disabled test step

gpt4all convert.py script not working

it says unsupported format on the source weights, some update may be needed or another set of weights

datalake for few shot prompting example is not saving data

the data set is created
the embeddings seems to be generated
but on datalake web inspector no data is shown

remove linter errors

the GitHub pipeline detects incomplete code