yushi-hu/ic-dst

Code base of In-Context Learning for Dialogue State tracking

ic-dst's Issues

multi-value slot values

Thanks for releasing code!

I'm trying to understand how this implementation handles multi-value slot values introduced in MultiWOZ 2.1 (such as e.g. cheap|moderate). It appears that this implementation considers a slot to be correctly predicted if at least one of gold slot values is predicted (looking at this code [here])(https://github.com/Yushi-Hu/IC-DST/blob/main/evaluate_metrics.py#L58). I did not see similar handling of multi-values in code releases for prior work (e.g. https://github.com/jshin49/ds2 or https://github.com/chiahsuan156/DST-as-Prompting or https://github.com/facebookresearch/Zero-Shot-DST/tree/main/TransferQA). Can you please comment on that?

DS2 T5 model is released now

Dear authors,

I saw that your paper experimented with our code so quickly, and I'm very happy to see such great progress from your work.

I released the T5 versions recently so if you want to compare it on 2.4, you can check our repository for the updates.

Note that our code used the 40GB NVIDIA Tesla A100 GPU (single GPU) to fine-tune the T5-large model so it may not work well with DDP with smaller ones, as we did not try.

Please let me know if there are any issues.

p.s. if you could modify the citations to the bibtex I posted in the code, that would be also much appreciated!

Best,
Jay