tsujuifu/pytorch_vsum-ptr-gan

A PyTorch implementation of VSumPtrGAN

[WACV'19 (Oral)] Attentive and Adversarial Learning for Video Summarization

A PyTorch implementation of VSumPtrGAN

Paper | Video

Overview

VSumPtrGAN is an implementation of
"Attentive and Adversarial Learning for Video Summarization"
Tsu-Jui Fu, Shao-Heng Tai, and Hwann-Tzong Chen
in IEEE Winter Conference on Applications of Computer Vision (WACV) 2019 (Oral)

VSumPtrGAN a GAN-based training framework, which combines the merits of unsupervised and supervised video summarization approaches. The generator is an attention-aware Ptr-Net that generates the cutting points of summarization fragments. The discriminator is a 3D CNN classifier to judge whether a fragment is from a ground-truth or a generated summarization. Our Ptr-Net generator can overcome the unbalanced training-test length in the seq2seq problem, and our discriminator is effective in leveraging unpaired summarizations to achieve better performance.

Requirements

This code is implemented under Python3 and PyTorch.
Following libraries are also required:

• PyTorch>=0.4
• FFmpeg

Usage

• VisualExtractor

Dataset/model_visual-extractor.ipynb

• VSumPtrGAN

model_vsum-ptr-gan.ipynb

Resources

• SumMe Dataset
• TVSum Dataset
• YouTube Dataset (Preprocessed)
• LoL Dataset

Citation

@inproceedings{fu2019vsum-ptr-gan, 
  author = {Tsu-Jui Fu and Shao-Heng Tai and Hwann-Tzong Chen}, 
  title = {{Attentive and Adversarial Learning for Video Summarization}}, 
  booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)}, 
  year = {2019} 
}

Acknowledgement

• Umbo Computer Vision
• pytorch_seq-gan

pytorch_vsum-ptr-gan's Issues

Am I missing the source

Hi,
Am I missing the source somwhow? or it is not uploaded yet?

Thanks

Ask about model_vsum-ptr-gan.ipyb

In the second section, I do not have a new folder(dataset_ytsum) in Dataset and I do not have summary.pkl after executing model_visual-extractor.ipynb.
Could you tell me where can I get summary.pkl?

feature = pickle.load(open('Dataset/dataset_ytsum/feature.pkl', 'rb')) summary = pickle.load(open('Dataset/dataset_ytsum/summary.pkl', 'rb'))

thanks,

ask dataset

hi,

Could you please share the lol dataset? The original link is dead

Thankyou

How is the summary.pkl created?

how to generate 'summary.pkl'

which line should be written that save discriminator model?

I've tried to execute and convert python code step by step.

I found that model don't save but load model in "model_vsum-ptr-gan.ipynb".

And Could you tell me how to inference model?
( just execute model_g.eval()? )

YouTube Dataset (Preprocessed) can't open

hi~
YouTube Dataset (Preprocessed) can't open，
could you upload a new Preprocessed Dataset in Google drive？
thanks you！

How can get change points using KTS?

I tried to get change points using KTS code.
But i couldn't get proper change points.

If someone get change points using KTS, please help me?

i've not good at result.

i've run code on jupyter-notebook step by step, and didn't edit codes.

when only generator model train, result is good.

however, when generator and discriminator model train, very bad.TT

After Generator&Discriminator train, Generator model evaluates.

How train model, i can get good result.

Thank you^^