This is the implementation of Generating Radiology Reports via Memory-driven Transformer at EMNLP-2020.
If you use or extend our work, please cite our paper at EMNLP-2020.
@inproceedings{chen-emnlp-2020-r2gen,
title = "Generating Radiology Reports via Memory-driven Transformer",
author = "Chen, Zhihong and
Song, Yan and
Chang, Tsung-Hui and
Wan, Xiang",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2020",
}
torch==1.7.1
torchvision==0.8.2
opencv-python==4.4.0.42
You can download the models we trained for each dataset from here.
We use two datasets (IU X-Ray and MIMIC-CXR) in our paper.
For IU X-Ray
, you can download the dataset from here and then put the files in data/iu_xray
.
For MIMIC-CXR
, you can download the dataset from here and then put the files in data/mimic_cxr
.
NOTE: The IU X-Ray
dataset is of small size, and thus the variance of the results is large.
There have been some works using MIMIC-CXR
only and treating the whole IU X-Ray
dataset as an extra test set.
Run bash train_iu_xray.sh
to train a model on the IU X-Ray data.
Run bash train_mimic_cxr.sh
to train a model on the MIMIC-CXR data.
Run bash test_iu_xray.sh
to test a model on the IU X-Ray data.
Run bash test_mimic_cxr.sh
to test a model on the MIMIC-CXR data.
Run bash plot_mimic_cxr.sh
to visualize the attention maps on the MIMIC-CXR data.