wmt-conference/wmt21-news-systems

WMT21 News Systems and Evaluations

This repository contains all primary submissions for the WMT21 news task, and the human evaluations. The submissions are in the following directories

• xml : One xml file for each language pair, containing source, reference(s) and hypotheses
• txt-ts : Sources, references and hypotheses in text files, including test suites
• txt : Same, but without test suites

Tools for extracting the raw text from the XML can be found here.

The human evaluations are in the humaneval directory.

wmt21-news-systems's Issues

What is the rid?

Hey, I wanted to ask what does the rid record in the to english (MTurk) evaluations? Let me know, thanks!

Difference between txt and txt-ts

Hi,

Thanks for sharing the data from WMT21. I had two queries:
(1) Is the dataset in txt-ts from the test set of the WMT21 News translation task or from the dev set?
(2) What is the difference between txt and txt-ts? The description mentions they are the same except 'test suites' but not sure what 'test suites' mean here. Can you please elaborate?

Thanks