rafaelvp-db / db-ancient-code-translation Goto Github PK

View Code? Open in Web Editor NEW

Simple repo showing code-to-code and code-to-text capabilities using LLMs on Databricks.

Python 67.20% Shell 32.80%

db-ancient-code-translation's Introduction

Databricks Ancient Code Translation

TLDR;

This repo demonstrates code translation (Code to Text, Code to Code) capabilities using Large Language Models (LLMs) on Databricks

Getting Started

Clone this repo into your Databricks Workspace
Configure a Databricks single node cluster with Databricks Runtime 13.2 for Machine Learning and an NVIDIA A100 GPU (A10 might also work, though with lower floating point precision)
- A100 Instances On Azure: Standard_NC24ads_A100_v4 instances
- A100 Instances On AWS: EC2 P4d instances
Install the following libraries into the cluster (you can also do it directly in the notebooks and leverage requirements.txt for that):

accelerate==0.21.0
ninja
alibi
einops
transformers
triton
xformers

Run the notebooks from the notebooks folder

Roadmap

~~PL/SQL~~
- ~~Generating code explanations (code to text)~~
- ~~Converting to PySpark~~
SAS
Snowflake

Authors

Reference

💫 StarCoder Github Project
Tunstall, Lewis and Lambert, Nathan and Rajani, Nazneen and Beeching, Edward and Le Scao, Teven and von Werra, Leandro and Han, Sheon and Schmid, Philipp and Rush, Alexander. Creating a Coding Assistant with StarCoder. Hugging Face Blog, 2023.
Big Science Open RAIL-M License

Appendix

Evaluation

To evaluate StarCoder and its derivatives, you can use the BigCode-Evaluation-Harness for evaluating Code LLMs.

Inference hardware requirements

In FP32 the model requires more than 60GB of RAM, you can load it in FP16 or BF16 in ~30GB, or in 8bit under 20GB of RAM with

# make sure you have accelerate and bitsandbytes installed
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bigcode/starcoder")
# for fp16 replace with  `load_in_8bit=True` with   `torch_dtype=torch.float16`
model = AutoModelForCausalLM.from_pretrained("bigcode/starcoder", device_map="auto", load_in_8bit=True)
print(f"Memory footprint: {model.get_memory_footprint() / 1e6:.2f} MB")