Fine-Tuned BERT-base on Squad v1.

[huggingface]

Fine-Tuned BERT-base on Squad v1.

[wasiahmad]

@thomwolf On SQuAD v1.1, BERT (single) scored 85.083 EM and 91.835 F1 as reported in their paper but when I fine-tuned BERT using run_squad.py I got {"exact_match": 81.0975, "f1": 88.7005}. Why there is a difference? What I am missing?

[thomwolf]

Thanks for the details.
This PyTorch repo is starting to be used by a larger community so we would have to be a little more precise than just rough numbers if we want to include such pre-trained weights.
If you want to add your weights to the repo, you should convert the weights in the PyTorch repo model and get evaluation results on SQuAD with the PyTorch model so everybody has a clean knowledge of what they are using. Otherwise I think it's better that people do their own training and know what are the capabilities of the fine-tuned model they are using.
Feel free to come back and re-open the issue if this something you would like to do.