Add Llama 3 support to `convert_llama_weights_to_hf()`

[calmitchell617]

Add Llama 3 support to `convert_llama_weights_to_hf()`

[amyeroberts]

cc @ArthurZucker

[calmitchell617]

I successfully performed the following steps to test whether adding Llama 3 support to this script would facilitate the use case I outlined above:

I downloaded Llama 2 in meta format (not HF format) with the torchtune cli. It is important not to test with a model in the HF format, because Llama 3 is not yet officially uploaded in HF format. It wouldn't be an apples to apples comparison.

CLI command used:

tune download meta-llama/Llama-2-7b-chat --output-dir <checkpoint_path> --hf-token $HF_TOKEN

I then fine tuned the downloaded meta-format Llama 2 with torchtune:

tune run \
    --nproc_per_node=4 \
    lora_finetune_distributed \
    --config llama2/7B_lora \
    batch_size=1 \
    seed=29 \
    tokenizer.path=<checkpoint_path> \
    checkpointer.checkpoint_dir=<checkpoint_path> \
    checkpointer.output_dir=<checkpoint_path> \
    dataset=torchtune.datasets.my_custom_dataset \
    checkpointer.checkpoint_files=['consolidated.00.pth'] \
    checkpointer=torchtune.utils.FullModelMetaCheckpointer \
    gradient_accumulation_steps=1 \
    lr_scheduler.num_warmup_steps=5 \
    enable_activation_checkpointing=False \
    dataset.max_rows=100 \
    epochs=1

Then I converted the fine-tuned model to format that can be loaded with from_pretrained() with the convert_llama_weights_to_hf() function. I named the script convert_checkpoint.py, but it just contains the convert_llama_weights_to_hf() function copy/pasted from Transformers:

python convert_checkpoint.py --input_dir <checkpoint_dir> --output_dir <output_dir>  --model_size 7B

I then ran a Gradio chatbot on the fine-tuned/converted model, and it worked as expected. So, it does seem that adding Llama 3 support to this script will unlock my particular use case (and probably many others).

[ArthurZucker]

see #30334

[calmitchell617]

Thank you! I will keep an eye on that on that PR.

[ArthurZucker]

Just needs the doc 🤗

[calmitchell617]

Good to know.

Let me know if that's something I could contribute, otherwise, thanks for the good work.