sail-sg/vgt

Video Graph Transformer for Video Question Answering (ECCV'22)

vgt's Issues

The links for pre-computed features for TGIF-QA (Trans) and TGIF-QA (FrameQA) are not working.

How to extract features from original video by myself?

Thank you for your excellent work! How can I extract the video features by myself?
Moreover, the link address of test video of NExT-QA is invalid. How can I find it?

Why did you use qmax=0 for training of Next-QA?

Object alignment

Hi, thank you for your great work.
Have you processed object alignment across frames (4 frames) per video clip (8 video clips in one data point) in your nextqa feat file?

features of TGIF dataset

Hello, I need the feature files for TGIF-QA.
Could you provide the features files like nexqa dataset in the h5 file, please?

Unable to get the features of TGIF-QA and MSRVTT-QA

Hello,Thank you very much for your work. I found that the links to the feature files for TGIF-QA and MSRVTT-QA have expired. Could you provide the feature files, please?

load pre-training weights to the model

Thanks for your insightful work.
I read that VGT has better performance with pre-training on WebVideo. I would like to know how to run the code to load the pre-trained weights you provide in WebVideo0.18M.

Low Test Accuracy

After downloading all the required files for nextqa, I executed the command :

./shells/next_test.sh 0

but found that the accuracy is only 19.47%.

And when I removed the line of "pretrain_path" in next_test.sh and executed the command aforementioned, I got the accuracy 20.04%.

So I wander whether fine-tuning is required for the given model "VGT_B5" on nextqa.

Val Accuracy

Thank you for sharing your programe. When I trained the model without CM-pretain under default setting, I got val accuracy of 54.22. How can I reproduce 55.02 which is presented in the paper? Thank you for your help!