support for using Huggingface Transformers GPT2, T5 and Bart implementation?

[nvidia]

support for using Huggingface Transformers GPT2, T5 and Bart implementation?

[byshiue]

1. If the GPT2 model of Huggingface transformer is same to the GPT2 paper, then yes.
2. If the model of T5 or Bart in Hugginface transformer is same to the decoder of Attention is all you need, then yes.

[gyin94]

• If the GPT2 model of Huggingface transformer is same to the GPT2 paper, then yes.
• If the model of T5 or Bart in Hugginface transformer is same to the decoder of Attention is all you need, then yes.

The layer names of the implementation would be different.

[byshiue]

• If the GPT2 model of Huggingface transformer is same to the GPT2 paper, then yes.
• If the model of T5 or Bart in Hugginface transformer is same to the decoder of Attention is all you need, then yes.

The layer names of the implementation would be different.

The layer name is used to get the tensor/weight. You can change them directly in python codes.

[gyin94]

Is there any plan to add the conversion since huggingface transformers are more popular and easy to use now? Thanks.

[byshiue]

We have such plan, but still not have enough bandwidth now.
We also encourage users to contribute their conversion.

[gyin94]

@byshiue I have another question. Have we added the log_softmax of logits for each token prediction in decoding? This can be useful for post processing. ctranslate2, lightseq and HuggingFace have all implemented it.

[byshiue]

If we need cumulate the previous probability (like beam search), we will use log softmax.