wietsedv/gpt2-recycle

As good as new. How to successfully recycle English GPT-2 to make models for other languages (ACL Findings 2021)

gpt2-recycle's Issues

Fine-tune the model on a custom dataset

Hello

Thanks very much for publishing this wonderful model, calibrated for Italian and Dutch.
I was wondering if there's the possibility to fine tune this model on a custom dataset.
If yes, could you show what is the input to give to the model?
Thanks very much

How to train a model

Question

Could you please the readme as to which files should be run and in which order to train a model for a new language.

Details on training of published models

Hi! I really enjoyed your paper and the code. Do you have some more details regarding the published Italian and Dutch models?

Specifically:

• for how many steps did you train only the embeddings?
• for how many steps did you train the full model?
• at which batch size / learning rate?

Sorry if this is already mentioned somewhere in the Repo or the paper, I wasn't able to find it. I am trying to use some of your results in my work, so this information would help a lot :) Thanks!