asappresearch/abcd

Official repository for "Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems"

abcd's Issues

The "do-eval" option will rewrite trained model

The second command in run.sh: main.py --do-eval --quantify --model-type roberta --prefix 0524 --filename final --task ast will rewrite the trained model checkpoint. Also it does not seem that the code has the ability to load a saved model.

issue reproducing the results

Hi there, I have ran the following command from run.sh, in an attempt to reproduce the results for roberta but I am facing discrepencie.

python main.py --learning-rate 1e-5 --weight-decay 0 --batch-size 3 --epochs 21 --log-interval 400 \
    --grad-accum-steps 4 --model-type roberta --prefix 0524 --filename final --task ast

I am getting these results, which seem different from Roberta in Table 4 where Value is 73.1% instead o 53.9.

• Could you explain this discrepency And quick question?
• is "intent" in Table 4 also called Action_Accuracy in the code? Thanks!