Huggingface checkpoints

[instadeepai]

Huggingface checkpoints

[ranzenTom]

Hi fteufel,

We are working on it with HuggingFace teams right now. For now, I would recommend using the code in this repository but the code on HuggingFace should be fully ready to be used in a few days. Right now the uploaded models are correct but the tokenizer is still missing and we are working on it.

[fteufel]

Thanks! This already helps a lot, as I'm having problems with jax. I'll use the tokenizer form the repo together with the HF weights.

[ranzenTom]

The 4 checkpoints and 2 pre-training datasets are now available on Huggingface, you can see the list here!

You can use them using standard classes like AutoTokenizer,AutoModelForMaskedLM or AutoModelForSequenceClassification. We fixed a couple of issues related to this in recent PRs, so please update transformers to the latest version on main with pip install --upgrade git+https://github.com/huggingface/transformers.git before trying to use them.

PyTorch only for now, but we expect TensorFlow versions to be available in a couple of days!