[BUG]

[thu-coai]

[BUG]

[zqwerty]

Thanks! We will check it in a few days.

[zqwerty]

Please check if you re-install ConvLab-2 since the code has changed. (pip install -e . in ConvLab-2)

[sherlock1987]

Yeah, I have already change the code, sure about that.

[zqwerty]

I have tried these:
in mle/multiwoz/: python train.py
in ppo/: python train.py --load_path ../mle/multiwoz/save/best_mle
in policy/: python evaluate.py --model_name PPO --load_path ppo/save/9_ppo (I choose 9_ppo randomly)
and get:

All 100 0.73
reward: 11.143058441558441

[sherlock1987]

Oh, that is cool, for me the PPO is also good, but for GDPL, there are still some problem exists.

[LinZichuan]

I have tried these:
in mle/multiwoz/: python train.py
in ppo/: python train.py --load_path ../mle/multiwoz/save/best_mle
in policy/: python evaluate.py --model_name PPO --load_path ppo/save/9_ppo (I choose 9_ppo randomly)
and get:

All 100 0.73
reward: 11.143058441558441

Hi @zqwerty , Thanks for the tips. This works for the commit 2422980!
But when I run these commands for the latest commit (c6372b1), I found the problem is still unsolved. The performance of PPO will still rise to 65% at the beginning and then start dropping at later training stage (to around 35%).
Could you help look into it? Thanks!

[zqwerty]

move to #54

[ShuoZhangXJTU]

I have tried these:
in mle/multiwoz/: python train.py
in ppo/: python train.py --load_path ../mle/multiwoz/save/best_mle
in policy/: python evaluate.py --model_name PPO --load_path ppo/save/9_ppo (I choose 9_ppo randomly)
and get:

All 100 0.73
reward: 11.143058441558441

Hey guys, I tested PPO on the latest version of convab-2 today and I got a success rate of 84% that is way bigger than the reported 73%, I wonder if there are any mistakes? If not, I think the performance record should be updated.