ictnlp/dstc8-avsd

We rank the 1st in DSTC8 Audio-Visual Scene-Aware Dialog competition. This is the source code for our IEEE/ACM TASLP (AAAI2020-DSTC8-AVSD) paper "Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog".

dstc8-avsd's Issues

The evaluation results i get are lower than reported in the paper

Hi, Thank you for sharing the code, but i have a little problem when reproducing the evaluation results like BLEU-1, BLEU-2, BLEU-3, METEOR. I use exactly the same code as you produced, and the generate result seems good. But when I use the evaluation tools produced in DSTC7-AVSD CHALLENGE or DSTC8-AVSD CHALLENGE, I only get Bleu_1: 0.687, Bleu_2: 0.562, CIDEr: 1.034, which are lower than the result in the paper(like Bleu_1: 0.759). Can you help me solve this problem or provide some pretrained model? Thanks.

it seems hard to reproduce your results in paper.

I tried to grid search the params in generate program, but the results can't achieve which in your paper. May I ask if it possible to share your SOTA CKPTS ?

Model Weights

Hi, can you release the model weights/checkpoint for the best submitted model?

'VideoGPT2Model' object has no attribute 'output_hidden_states'

Hi,

I am using pytorch 1.7.1 and getting the following error:

torch.nn.modules.module.ModuleAttributeError: 'VideoGPT2Model' object has no attribute 'output_hidden_states'

Question on reproduction

Thank you for sharing your work. I cannot reproduce the results of the paper in practice. May I ask how many epochs were trained during training and how much the final loss converged?