CE scores

[Yingshu-Li]

CE scores

[wjhou]

Hi,

We use macro-weighted metrics for evaluation. You may also read the comment from this issue zhjohnchan/R2GenCMN#12.

Best,
Ethan

[Yingshu-Li]

Hi,

I have tried your mentioned method. I used chexpert to label the report. I cannot reproduce the result. The scores are much lower.

here is my method:

1. use chexbert to label the report. https://github.com/stanfordmlgroup/CheXbert
2. set label: nan and -1 to 0.
3. use this script to calculate the score. https://github.com/zhjohnchan/R2GenCMN/blob/main/compute_ce.py

Could you please help me to check if I have made any mistakes?

best

[wjhou]

Hi,

We set -1 (i.e., Uncertain) to 1 in the calculation, similar to https://github.com/zzxslp/WCL/blob/main/chexpert-labeler/calculate_metric.py.

Could you provide the scores (NLG and CE scores) you obtained?

Best,
Ethan