hsinyinglee/opn

opn's Issues

Missing test data

The test data has not been included in your project page. Can you please provide links to the test data matrices for the UCF and HMDB datasets?

Motion-aware tuple selection

Hi, I am very interested in the data preprocessing part, cause it's very important to avoid trivial problem.
And it seems that Motion-aware tuple selection part is not released. Could you please release this part of code or give the detail steps of motion-aware extraction step?

Many thanks!

Finetuning scheme

How do you finetune on the action recognition datasets? The datasets contain videos. Do you randomly extract frames from the videos? What is the size of the frames that you use?

OPN Model testing

Hi @HsinYingLee , I would like to thank you for the tutorial.

I have a question about applying an already trained model (Unsupervised trained on UCF) or (Unsupervised trained on UCF+HMDB+ACT) on an external video in order to re-sort its frames sequence using:
$ python visualize.py $MODEL $OUTPUT_FIG
so how can define the video sequence directory?
what does "$OUTPUT_FIG" represent?

Thanking you in advance.

pre-trained model size mismatch

Hi Hsin Ying Lee,

The pre-trained model has 24 output but the model in train_opn.prototxt has only 12 output. I think that 4 images have 24 orders and thus the number of orders should be 24. Thus the 12 orders in train_opn.prototxt and UCF_datalayers.py may be wrong.

Zhangjie

Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0)

Hello,

When I run the code, it throws this error:

Creating Layer conv1
I1011 13:25:12.462865  7866 net.cpp:406] conv1 <- im_concat
I1011 13:25:12.462875  7866 net.cpp:380] conv1 -> conv1
F1011 13:25:12.899116  7866 cudnn.hpp:122] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0)  CUDNN_STATUS_BAD_PARAM
*** Check failure stack trace: ***
    @     0x7f01584a5daa  (unknown)
    @     0x7f01584a5ce4  (unknown)
    @     0x7f01584a56e6  (unknown)
    @     0x7f01584a8687  (unknown)
    @     0x7f0158b6c8f3  caffe::CuDNNConvolutionLayer<>::Reshape()
    @     0x7f0158af40fa  caffe::Net<>::Init()
    @     0x7f0158af6382  caffe::Net<>::Net()
    @     0x7f0158c08020  caffe::Solver<>::InitTrainNet()
    @     0x7f0158c08583  caffe::Solver<>::Init()
    @     0x7f0158c0885f  caffe::Solver<>::Solver()
    @     0x7f0158c16b61  caffe::Creator_SGDSolver<>()
    @           0x40d187  caffe::SolverRegistry<>::CreateSolver()
    @           0x4083be  train()
    @           0x405c7c  main
    @     0x7f0156cf4f45  (unknown)
    @           0x4065ab  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)

Do you think it's a version issue? I have cudnn version 5 and cuda version 8. My system is Ubuntu 14.04.

Thanks.

Missing validation data

Hello,

I could not find "UCF_val.mat" file in your project page. This throws an error when I try to run the code.

Please share that file. Thanks!

Combine the information of different frames for Action Recognition

Hi Hsin-Ying,

Thank you for your code. I hope to use your network in our work. Yet I am not quite clear how the network is finetuned for Action Recognition task in your paper. It would be very helpful if you could clear my confusion.

To predict the action of a clip, how did you combine the features of all the clip's frames?
For example, did you use the features of the last conv layer and concatenate the features of all frames then feed them to FC layers? Or did you sample only one frame from a clip and classify actions based on the selected frame?

Thanks! I saw you had answered similar questions. Yet I am still confused.

Finetune process

Hello HsinYing, nice work!
I am wondering how do you finetune the unsupervised UCF101 dataset. In your paper, you report mean classification accuracy over the 3 splits of UCF101 dataset. Could you please describe it more?

In my understanding, after training with unlabeled video from UCF101 training split 1, you first finetune the pre-trained network with test split 1 from UCF101 with label, and then you can get the accuracy of split 1. And again, finetune the pre-trained network with test split 2 from UCF101 with label, get the accuracy of split 2; and same as the split3. And after average them together, the mean accuracy is obtained.

Thanks a lot!