CMRC 2018 Official Website (Chinese only)：https://cmrc2018.hfl-rc.com/
The Second Evaluation Workshop on Chinese Machine Reading Comprehension was succesfully ended. The evaluation committee had decided to continue to accept submissions to further evaluations on the hidden test set and challenge set.
CMRC 2018 Public Datasets: https://worksheets.codalab.org/worksheets/0x92a80d2fab4b4f79a2b4064f7ddca9ce
Submission Guidelines: https://worksheets.codalab.org/worksheets/0x96f61ee5e9914aee8b54bd11e66ec647/
Open Challenge Leaderboard: https://hfl-rc.github.io/cmrc2018/open_challenge/
2019/3/29 We provide SQuAD-style CMRC 2018 datasets, which is exactly the same with the format for SQuAD, see CodaLab link
2018/12/7 Open Challenge has been announced.
2018/10/18 System overview paper was out.
2018/5/7 Training and development data has been released.
2018/3/13 Trial data has been released.
2018/2/1 The trial data will be available on March 5, 2018.
If you are participating CMRC 2018, please download data and evaluation script through CodaLab.
System overview: https://arxiv.org/abs/1810.07366
If you wish to use this data in your research, please cite:
@article{cmrc2018-dataset,
title={A Span-Extraction Dataset for Chinese Machine Reading Comprehension},
author={Cui, Yiming and Liu, Ting and Xiao, Li and Chen, Zhipeng and Ma, Wentao and Che, Wanxiang and Wang, Shijin and Hu, Guoping},
journal={arXiv preprint arXiv:1810.07366},
year={2018}
}
ISLRN: 013-662-947-043-2
http://www.islrn.org/resources/resources_info/7952/
The First Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC2017) was a great success co-located with the CCL2017 at Nanjing on October, 2017. The CMRC2017 has attracted lots of attention from the Chinese NLP community. We would like to express our sincere thanks to all the participants and the support from the community.
To further accelerate the progress of Chinese Machine Reading Comprehension field, we are going to organize The Second Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC2018) this year, and will be co-located with the CCL2018 at Changsha on October 19 ~ 21.
CMRC2018 is hosted by the Technical Committee on Computational Linguistics, Chinese Information Processing Society of China (CIPS-CL), organized by Joint Laboratory of HIT and iFLYTEK Research, sponsored by iFLYTEK Co., Ltd.. We aim to provide a platform for the related researchers and a forum for communications on the related research. Welcome to join us!
Joint Laboratory of HIT and iFLYTEK Research（HFL） is devoting for the development and research on the machine reading comprehension. On Chinese reading comprehension, HFL has released the first Chinese cloze-style reading comprehension dataset: PD&CFT. In 2017, HFL has organized the first evaluation workshop on Chinese machine reading comprehension, which accelerated the research on Chinese reading comprehension. Through annual Chinese Machine Reading Comprehension workshop, we hope the researchers on related field could jointly promote the technical level of Chinese machine reading comprehension.
In the last year, we focus on the Cloze-style reading comprehension task, and attracted many participants on evaluation. This year, we will focus on the Span-Extraction Machine Reading Comprehension, which is a extension of cloze-style reading comprehension. We have seen many dataset on this kind of reading comprehension, such as SQuAD, NewsQA. However, we did not see there is a Chinese corpus for this purpose. To add diversity in Chinese dataset, we will release the first Chinese Span-Extraction dataset. The participants will analyze the context and query and extract the correct span in the context for answer output.
Following the rule in previous evaluation, we will release training and validation set at first and keep the test set hidden for fairness of the evaluation process.
We will award the top-3 systems as well as the best single model system on our evaluation. The details can be illustrated as follows.
| Gold Prize | ￥20,000 + Certificate* |
|---|---|
| Silver Prize | ￥10,000 + Certificate |
| Bronze Prize | ￥5,000 + Certificate |
| Best Single System Prize | ￥10,000 + Certificate |
*The certificate is provided by CIPS-CL
All the deadlines are Beijing Time (GMT+8) 23:59.
PLEASE PAY CLOSE ATTENTION TO THE WEBSITES, IN CASE THERE WILL BE CHANGE IN DEADLINE
| Process | State | Time |
|---|---|---|
| Pre-registration | END | From now on |
| Release of trial data | END | March 5, 2018 |
| Confirmation for registration | END | April 23, 2018 ~ April 27, 2018 |
| Release of Training and development set | END | May 7, 2018 |
| Tuning System | May 7, 2018 ~ August 7, 2018 | |
| Validation for development set | END | June 7, 2018 ~ August 7, 2018 |
| Submission for final system | END | August 13, 2018 ~ August 17, 2018 |
| System Description | END | Mid-September, 2018 |
| CMRC2018 workshop | >>Running<< | TBD, co-locate with CCL2018（October 20 or 21, 2018） |
Please fill the following form for participation: https://wj.qq.com/s/1822356/e14c
Organizer： Joint Laboratory of HIT and iFLYTEK Research
Sponsor： iFLYTEK Co., Ltd.
Evaluation Committee：
Ting Liu, Harbin Institute of Technology
Yiming Cui, iFLYTEK Research
Follow Joint Laboratory of HIT and iFLYTEK Research (HFL) on WeChat.
Any problems? Feel free to concat us.
E-MAIL： [email protected]
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent