This repository contains multiple components related to medical report summarization and evaluation. It includes both baseline and proposed methods for generating and evaluating reports, as well as test datasets.
-
BART-summarization
- This directory contains the code for the baseline report summarization using BART. It focuses on generating summaries from medical reports.
-
BART-our-proposed
- This directory includes our proposed method for report summarization, which is based on the BART-large model. It contains enhancements over the baseline to improve the quality of the generated summaries.
-
R2Gen-main
- Contains the code for the baseline report generation approach named R2Gen. This serves as a baseline method for generating medical reports from the given data.
-
R2Gen-our-proposed
- Includes the code for our proposed improvements to the R2Gen model, aimed at enhancing the quality and accuracy of the generated reports.
-
report_generation_baseline_results
- This folder contains the results from the baseline report generation methods. It includes the generated reports which can be compared against the ground truth or other models.
-
report_summarization_baseline_results
- This folder contains the results from the baseline report summarization methods. It includes the generated reports which can be compared against the ground truth or other models.
-
chexpert-evaluation
- Contains the evaluation scripts for assessing the Chexpert score of the generated medical reports. The Chexpert score is used to evaluate the clinical relevance and correctness of the generated content.
-
test_image.json
- This file is the test dataset used in the evaluation. Each line in the file is a JSON object containing the input
findings
and the corresponding outputimpression
generated by the models.
- This file is the test dataset used in the evaluation. Each line in the file is a JSON object containing the input
Report Generation: For R2Gen-main
and R2Gen-our-proposed
, the train code is
python main_train.py \
--image_dir no_use \
--ann_path annotation.json \
--dataset_name mimic_cxr \
--max_seq_length 100 \
--threshold 10 \
--batch_size 16 \
--epochs 30 \
--save_dir results/RGen14 \
--step_size 1 \
--gamma 0.8
Evaluation: To decode and evaluate the model, use the following command:
python main_test.py \
--image_dir data/mimic_cxr/images/ \
--ann_path annotation.json \
--dataset_name mimic_cxr \
--max_seq_length 100 \
--threshold 10 \
--batch_size 16 \
--epochs 30 \
--save_dir results/RGen \
--step_size 1 \
--gamma 0.8 \
--load checkpoint.pth
Report Summarization: For BART-summarization
and BART-our-proposed
, the train code is
python3 run_mybart.py --model_name_or_path facebook/bart-base \
--do_train --output_dir das \
--exp_name exp_name \
--max_source_length 1024 --max_target_length 300 \
--save_dataset_path withlabel\
--num_train_epochs 100 \
--per_device_train_batch_size 8 --save_strategy epoch \
--label_smoothing_factor 0.1 --weight_decay 0.01 \
--max_grad_norm 0.1 --warmup_steps 500\
--gradient_accumulation_steps 8 \
--lr_scheduler_type polynomial --learning_rate 3e-05
Evaluation: To decode and evaluate the model, use the following command:
CUDA_VISIBLE_DEVICES=0 python3 run_mybart.py \
--per_device_eval_batch_size 64 \
--log_root ./log \
--save_dataset_path $dataset \
--exp_name $proj_name \
--do_predict --predict_with_generate True \
--output_dir das \
--val_max_target_length 100 \
--model_name_or_path checkpoint
Use the chexpert-evaluation
directory to evaluate the Chexpert score of the generated reports, which helps in assessing their clinical relevance.