Request for Training Code for Reproducibility and Research Purposes

[google-research]

Request for Training Code for Reproducibility and Research Purposes

[Serena-LiuWx]

Thanks a lot for your advice!

[shayne-longpre]

Hi @Serena-LiuWx thank you for your kind words and interest! Unfortunately, it is not within my power to release all the experiment code, as much as I would have liked to. It was a huge push to get the (significant) approvals required just to re-create an external version of the data processing pipeline you now see here.

However, these experiments should not be hard to reproduce using public infrastructure like seqio and T5X. There are also a lot of repositories that have released instruction finetuning code on top of many models, including T5 (e.g. T0). You should be able to swap out the training data and make it work.

Hopefully this helps, and best of luck!

[StephennFernandes]

is the training code out yet ?