dahan5/falcon-40b-7b

Using Falcon40B or Falcon7B Instruct or any other Open Source LLMs on GPU via HuggingFace

Using Falcon40B or Falcon7B Instruct or any other Open Source LLMs on GPU via HuggingFace

In this tutorial we're going to be checking out some of the biggest baddest LLMs...but running them on a GPU!

Startup 🚀

1. Create a virtual environment python -m venv gpullm
2. Activate it:
   • Windows:.\gpullm\Scripts\activate
   • Mac: source gpullm/bin/activate
3. Install PyTorch with CUDA Support N.B. I've included the lib in the requirements.txt file but this is latest installer as of creating this readme. pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 Download and install CUDA (i've used 11.7 for this tutorial): https://developer.nvidia.com/cuda-11-7-0-download-archive Download and install the matching cuDNN version ( v8.9.1): https://developer.nvidia.com/rdp/cudnn-archive
4. Clone this repo git clone https://github.com/dahan5/falcon-40b-7b
5. Go into the directory cd falcon-40b-7b
6. Startup jupyter by running jupyter lab in a terminal or command prompt
7. Hit Ctrl + Enter to run through the notebook!
8. Go back to my YouTube channel and like and subscribe 😉...no seriously...please! lol

Other References 🔗

-PyTorch Installation:main guide leveraged to handle GPU support.

-Langchain HF Pipelines:the HF Pipelines class is used in order to pass the local LLM to a chain.

-Falcon 40B Instruct Model Card:check out the model details.

-Falcon 7B Instruct Model Card:check out the model details.

Who, When, Why?

👨🏾‍💻 Author: Taleb DAHAN
📅 Version: 1.x
📜 License: This project is licensed under the MIT License