sangram6n/open-llms

🤖 A list of open LLMs available for commercial use.

Open LLMs

These LLMs are all licensed for commercial use (e.g., Apache 2.0, MIT, OpenRAIL-M). Contributions welcome!

Language Model	Release Date	Checkpoints	Paper/Blog	Size	Context Length	Licence
T5	October 2019	T5 & Flan-T5, Flan-T5-xxl (HF)	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	60M - 11B	512	Apache 2.0
UL2	October 2022	UL2 & Flan-UL2, Flan-UL2 (HF)	UL2 20B: An Open Source Unified Language Learner	20B	512, 2048	Apache 2.0
Cerebras-GPT	March 2023	Cerebras-GPT	Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models (Paper)	111M - 13B	2048	Apache 2.0
Open Assistant (Pythia family)	March 2023	OA-Pythia-12B-SFT-8, OA-Pythia-12B-SFT-4, OA-Pythia-12B-SFT-1	Democratizing Large Language Model Alignment	12B	2048	Apache 2.0
Pythia	April 2023	pythia 70M - 12B	Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling	70M - 12B	2048	Apache 2.0
Dolly	April 2023	dolly-v2-12b	Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM	3B, 7B, 12B	2048	MIT
RWKV	August 2021	RWKV, ChatRWKV	The RWKV Language Model (and my LM tricks)	100M - 14B	infinity (RNN)	Apache 2.0
GPT-J-6B	June 2023	GPT-J-6B, GPT4All-J	GPT-J-6B: 6B JAX-Based Transformer	6B	2048	Apache 2.0
GPT-NeoX-20B	April 2022	GPT-NEOX-20B	GPT-NeoX-20B: An Open-Source Autoregressive Language Model	20B	2048	Apache 2.0
Bloom	November 2022	Bloom	BLOOM: A 176B-Parameter Open-Access Multilingual Language Model	176B	2048	OpenRAIL-M v1
StableLM-Alpha	April 2023	StableLM-Alpha	Stability AI Launches the First of its StableLM Suite of Language Models	3B - 65B	4096	CC BY-SA-4.0
FastChat-T5	April 2023	fastchat-t5-3b-v1.0	We are excited to release FastChat-T5: our compact and commercial-friendly chatbot!	3B	512	Apache 2.0
h2oGPT	May 2023	h2oGPT	Building the World’s Best Open-Source Large Language Model: H2O.ai’s Journey	12B - 20B	256 - 2048	Apache 2.0
MPT-7B	May 2023	MPT-7B, MPT-7B-Instruct	Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs	7B	84k (ALiBi)	Apache 2.0, CC BY-SA-3.0
RedPajama-INCITE	May 2023	RedPajama-INCITE	Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models	3B - 7B	2048	Apache 2.0
OpenLLaMA	May 2023	OpenLLaMA-7b-preview-300bt	OpenLLaMA: An Open Reproduction of LLaMA	7B	2048	Apache 2.0

LLMs for code

Language Model	Checkpoints	Paper/Blog	Size	Context Length	Licence
SantaCoder	santacoder	SantaCoder: don't reach for the stars!	1.1B	2048	OpenRAIL-M v1
StarCoder	starcoder	StarCoder: A State-of-the-Art LLM for Code, StarCoder: May the source be with you!	15B	8192	OpenRAIL-M v1
StarChat Alpha	starchat-alpha	Creating a Coding Assistant with StarCoder	16B	8192	OpenRAIL-M v1
Replit Code	replit-code-v1-3b	Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit	2.7B	infinity? (ALiBi)	CC BY-SA-4.0
CodeGen2	codegen2 1B-16B	CodeGen2: Lessons for Training LLMs on Programming and Natural Languages	1B - 16B	2048	Apache 2.0

Evals on open LLMs

• Leaderboard by lmsys.org
• Evals by MosaicML
• Holistic Evaluation of Language Models (HELM)
• LLM-Leaderboard

LLM datasets for fine-tuning

PENDING

Want to contribute? Just add a row above.

---

What do the licences mean?

• Apache 2.0: Allows users to use the software for any purpose, to distribute it, to modify it, and to distribute modified versions of the software under the terms of the license, without concern for royalties.
• MIT: Similar to Apache 2.0 but shorter and simpler. Also, in contrast to Apache 2.0, does not require stating any significant changes to the original code.
• CC BY-SA-4.0: Allows (i) copying and redistributing the material and (ii) remixing, transforming, and building upon the material for any purpose, even commercially. But if you do the latter, you must distribute your contributions under the same license as the original. (Thus, may not be viable for internal teams.)
• OpenRAIL-M v1: Allows royalty-free access and flexible downstream use and sharing of the model and modifications of it, and comes with a set of use restrictions (see Attachment A)

Disclaimer: The information provided in this repo does not, and is not intended to, constitute legal advice. Maintainers of this repo are not responsible for the actions of third parties who use the models. Please consult an attorney before using models for commercial purposes.

---

Improvements

• Complete entries for context length, and check entries with ?
• Add number of tokens trained? (see considerations)
• Add (links to) training code?
• Add (links to) eval benchmarks?