bbrattoli/zeroshotvideoclassification

Zero-shot video classification by end-to-end training of 3D convolutional neural networks

zeroshotvideoclassification's Issues

About KINETICS class numbers

Hi,
Thanks for your great work.
I run the code and find that there are only 662 classes with a 0.05 overlap threshold.
This is different from the class number (664) reported in the paper.
Could you help me fix this problem?

KINETICS: total number of videos 558278, classes 700
After filtering) KINETICS: total number of videos 521647, classes 662

KeyError: "word '---e1gyo84' not in vocabulary"

Sorry to bother you, but I have encountered the following problem:
python3 main.py --n_epochs 150 --bs 22 --lr 1e-3 --network c3d --dataset kinetics2both --save_path /home/m/Desktop/ZeroShotVideoClassification-master/result --nopretrainedTotal batch size: 22
UCF101: total number of videos 13320, classes 101
HMDB51: total number of videos 6766, classes 51
Traceback (most recent call last):
File "main.py", line 66, in
dataloaders = dataset.get_datasets(opt)
File "/home/m/Desktop/ZeroShotVideoClassification-master/dataset.py", line 14, in get_datasets
get_datasets = get_both_datasets(opt)
File "/home/m/Desktop/ZeroShotVideoClassification-master/dataset.py", line 109, in get_both_datasets
train_class_embedding = classes2embedding('kinetics', train_classes, wv_model)
File "/home/m/Desktop/ZeroShotVideoClassification-master/auxiliary/auxiliary_word2vec.py", line 20, in classes2embedding
embedding = [one_class2embed(class_name, wv_model)[0] for class_name in class_name_inputs]
File "/home/m/Desktop/ZeroShotVideoClassification-master/auxiliary/auxiliary_word2vec.py", line 20, in
embedding = [one_class2embed(class_name, wv_model)[0] for class_name in class_name_inputs]
File "/home/m/Desktop/ZeroShotVideoClassification-master/auxiliary/auxiliary_word2vec.py", line 119, in one_class2embed_kinetics
return wv_model[name_vec].mean(0), name_vec
File "/home/m/Anaconda/envs/pytorch/lib/python3.6/site-packages/gensim/models/keyedvectors.py", line 355, in getitem
return vstack([self.get_vector(entity) for entity in entities])
File "/home/m/Anaconda/envs/pytorch/lib/python3.6/site-packages/gensim/models/keyedvectors.py", line 355, in
return vstack([self.get_vector(entity) for entity in entities])
File "/home/m/Anaconda/envs/pytorch/lib/python3.6/site-packages/gensim/models/keyedvectors.py", line 471, in get_vector
return self.word_vec(word)
File "/home/m/Anaconda/envs/pytorch/lib/python3.6/site-packages/gensim/models/keyedvectors.py", line 468, in word_vec
raise KeyError("word '%s' not in vocabulary" % word)
KeyError: "word '---e1gyo84' not in vocabulary"

I look forward to your reply. Thank you very much

module 'dataset' has no attribute 'get_dataloaders'

dataloaders = dataset.get_dataloaders(opt)

The data loading method here is not seen in dataset.

Pretrained models

Thanks for your great work.
Do you have plans to release pretrained model weights? This would be a significant addition to the ZSL field.

Thanks,

Number of V100s used for training

Hello

Thank you for the work :)

May I ask the number of V100s you used for training the model?

Trying to estimate the total batch size you used (understand that its 22 per V100 GPU)

Would you give some instructions to download and adapt to the datasets?

Would I download all datasets for zero-shot video classification?

did you ever try same training using BERT or similar model instead of simple Word2Vec

Hi, I follow your work and this is a great work, very simple and effective :)
I am wondering did you try or know of similar training with Bert or a similar transformer model; I am trying something like that, but the loss seems to remain fairly steady, and the model is not learning anything. The same framework is working fine with word2vec, Do you know why this may happen? any intuitive thought?
@bbrattoli