feyzaakyurek/dune

Dataset for Unified Editing, EMNLP 2023. This is a model editing dataset where edits are natural language phrases.

dune's Issues

Inquiry on Debiasing Evaluation

Hi @feyzaakyurek,

Apologies for reaching out again, but I have a couple of questions:

1. Could you provide a shell script for evaluating the debiasing process? I found that there are only arithmetic, newinfo and scientific under the folder script.
2. In dune/debiasing_I.json and dune/debiasing_II.json, under test_inputs, are the answers after the questions the ground truth? If so, it seems odd because the edits appear debiased, but the answers don't reflect this.

Thank you for your help!

Missing `arc_processed.csv` in `source/arc/` directory

Hello,

Thanks for your excellent work! I've been reviewing the ARC class in the EditDataset module and noticed a reference to the file arc_processed.csv within the source/arc/ directory. However, it appears that this file is missing. I was also unable to locate some files for other datasets.

Could you please clarify if there's an issue with the file locations or if they're supposed to be generated during the dataset processing? Any guidance on how to resolve this and ensure all necessary files are present would be greatly appreciated.

Thank you!