akanyaani/minillama

A simplified LLAMA implementation for training and inference tasks.

👋 Welcome to My GitHub Profile!

I'm Abhay Kumar, Principal Research Scientist at Yellow.ai, with a deep passion for training neural networks, particularly language models and large language models (LLMs). With over 8 years of experience in Data Science and NLP.

🌟 Highlights

• Principal Research Scientist at Yellow.ai
• Co-Author of Komodo LLM: An open-source, language-specific LLM for Indonesian.
• 8+ years of experience in Data Science and NLP, with a focus on Language Modeling.

💡 Projects

• Komodo LLM: A foundational large language model tailored for a specific language.
• GPT2-TF: Implementation of GPT-2 in TensorFlow 2, recognized as the first repository for GPT-2 in TensorFlow 2.
• miniLLama: A straightforward and compact implementation of the LLAMA Model, inspired by Andrej Karpathy's minGPT.

minillama's Issues

binarize the dataset and save to a file like: train.bin, val.bin instead of re binarize dataset from scratch