openhackathons-org/end-to-end-llm

This repository is an AI Bootcamp material that consist of a workflow for LLM

End-to-End LLM Bootcamp

The End-to-End LLM (Large Language Model) Bootcamp is designed from a real-world perspective that follows the data processing, development, and deployment pipeline paradigm. Attendees walk through the workflow of preprocessing the SQuAD (Stanford Question Answering Dataset) dataset for Question Answering task, training the dataset using BERT (Bidirectional Encoder Representations from Transformers), and executing prompt learning strategy using NVIDIA® NeMo™ and a transformer-based language model, NVIDIA Megatron. Attendees will also learn to optimize an LLM using NVIDIA TensorRT™, an SDK for high-performance deep learning inference, guardrail prompts and responses from the LLM model using NeMo Guardrails, and deploy the AI pipeline using NVIDIA Triton™ Inference Server, an open-source software that standardizes AI model deployment and execution across every workload.

Bootcamp Content

This content contains three Labs, plus an introductory notebook and two lab activities notebooks:

• Overview of End-To-End LLM bootcamp
• Lab 1: Megatron-GPT
• Lab 2: TensorRT-LLM and Triton Deployment with LLama-2-7B Model
• Lab 3: NeMo Guardrails
• Lab Activity 1: Question Answering task
• Lab Activity 2: P-tuning/Prompt tuning task

Tools and Frameworks

The tools and frameworks used in the Bootcamp material are as follows:

• NVIDIA NeMo™
• NVIDIA TensorRT™
• NVIDIA Triton™ Inference Server

Tutorial duration

The total Bootcamp material would take approximately 8 hours and 45 minutes. We recommend dividing the material's teaching into two days, covering Lab 1 in one session and the rest in the next session.

Deploying the Bootcamp Material

To deploy the Labs, please refer to the Deployment guide presented here

Attribution

This material originates from the OpenHackathons Github repository. Check out additional materials here

Don't forget to check out additional Open Hackathons Resources and join our OpenACC and Hackathons Slack Channel to share your experience and get more help from the community.

Licensing

Copyright © 2023 OpenACC-Standard.org. This material is released by OpenACC-Standard.org, in collaboration with NVIDIA Corporation, under the Creative Commons Attribution 4.0 International (CC BY 4.0). These materials may include references to hardware and software developed by other entities; all applicable licensing and copyrights apply.