ehsk/openqa-eval

ACL 2023: Evaluating Open-Domain Question Answering in the Era of Large Language Models

openqa-eval's Issues

Odd question formatting with NQ301_text-davinci-003.tsv

The questions in NQ301_text-davinci-003.tsv are formatted in a different way than in the other files which makes it much harder to use this data.

I've noticed at least the following issues:

1. They end in question marks that the other questions don't have
2. Apostrophes are formatted as a single quote with spaces on both sides while other questions use the ’ character without extra spaces
3. One question splits "ain't" into "ai n ' t"

Fixing this is probably pretty straightforward if you just replace the questions from NQ301_text-davinci-003.tsv with them from any of the other NQ301 files. As far as I can see they're in the same order (with the same ids) for all of them.