wanglimin/tdd

Trajectory-pooled Deep-Convolutional Descriptors

tdd's Issues

The value of the input named 'power' of extract_fv.m

Hi, @wanglimin
I want to reproduce the experiment result in the paper.

And I found a parameter named 'power' in extract_fv.m. The use of this para is setting feature to feature.^power before coding.

What's the value of this para should I set in order to reproduce the result?

Thanks.

script_demo.m problem

Hi, LiMin.
What's the meaning of sizes in line 19 of script_demo

sizes = [8,8; 11.4286,11.4286; 16,16; 22.8571,24;32,34.2587];%why?

In my opinion, according to your paper, sizes shold be

 [16*1/2,16*1/2; 16*1/sqrt(2), 16*1/sqrt(2); 16,16;16*sqrt(2),16*sqrt(2);16*2,16*2]

Am I misunderstanding something?

Unbalanced category accuracy in HMDB51

Hi, @wanglimin

Have you encountered the problem of unbalanced category accuracy while training and testing on HMDB51 dataset? For example, the accuracy of the class "kick" is much better than the class "brush_hair" in HMDB51 dataset while testing.

Thanks,
Philip

FlowCNNFeature.m

Hi, LiMin
The code in FlowCNNFeature.m

function FCNNFeature = FlowCNNFeature(vid_name, use_gpu, NUM_HEIGHT, NUM_WIDTH, model_def_file, model_file, gpu_id)
L = 10;
% Input video
filelist =dir([vid_name,'*_x*.jpg']);
if length(filelist) > 30 *60
    video = zeros(NUM_HEIGHT,NUM_WIDTH,L*2,30*60,'single');
else
    video = zeros(NUM_HEIGHT,NUM_WIDTH,L*2,length(filelist),'single');
end

for i = 1:size(video,4)
    flow_x = imread(sprintf('%s_%06d.jpg',[vid_name,'flow_x'],i));
    flow_y = imread(sprintf('%s_%06d.jpg',[vid_name,'flow_y'],i));
    video(:,:,1,i) = imresize(flow_x,[NUM_HEIGHT,NUM_WIDTH],'bilinear');
    video(:,:,2,i) = imresize(flow_y,[NUM_HEIGHT,NUM_WIDTH],'bilinear');
end

for i = 1:L-1% 
    tmp = cat(4, video(:,:,(i-1)*2+1:i*2,2:end),video(:,:,(i-1)*2+1:i*2,end));%240*320*2*54
    video(:,:,i*2+1:i*2+2,:)  = tmp;
end

I find it hard to understand the code

for i = 1:L-1% 
    tmp = cat(4, video(:,:,(i-1)*2+1:i*2,2:end),video(:,:,(i-1)*2+1:i*2,end));%
    video(:,:,i*2+1:i*2+2,:)  = tmp;
end

For example there are 55 frames in a video, so the variable video's shape is (240 320 20 55).
For every frame, there are 20 flow images with size 240*320 corresponds to it.
According to the code, for every frame, the first two flow image are flow_x and flow_y, the remaining 18 flow images are just the copy of the first two?
Does this make sense?

Why not just use

 tmp =video(:,:,(i-1)*2+1:i*2,1:end);

classification issue

Hi @wanglimin

1. For each video , the final dim of features is 262144 (4 * 65536), am i right?(after concatenating all layers' features )
   So, the dim for the training file in HMDB51 is 3570 * 262144,
   the dim for the testing file in HMDB51 is 1530 * 262144 .Am i right ? .
   You mentioned in your paper "then use a linear SVM as the classifer (C = 100)"
   It consumes a lot of time to train a file with 3570*262144 using libsvm library ,am i right ?
2. You mentioned "combining TDDs with iDTs, using early fusion of Fisher vector representation"
   How do you make the "early fusion" please ?

Thanks
Raymond

flow_mean.mat all zero?

The elements of flow_mean.mat are all zeros?

some confusion withextract_fv.m

Hi, @wanglimin
I want to know how to generate the ‘PCA’ in extract_fv.m

Thanks

matcaffe error

Hi, LiMin
When I run script_demo.m

>> script_demo
Extract improved trajectories...
video size, length: 55, width: 320, height: 240
Extract TVL1 optical flow field...
Extract spatial TDD...
Undefined function 'caffe' for input arguments of type 'char'.

Error in RGBCNNFeature (line 18)
if caffe('is_initialized') == 0

Error in script_demo (line 32)
feature_conv = RGBCNNFeature(vid_name, 1, sizes_vid(scale,1), sizes_vid(scale,2), model_def_file, model_file, gpu_id);

what's wrong? I am using the latest version of caffe.
Thank you very much!

spatial_cls.prototxt

这里pad不应该是2？

_The second modification is that before each convolutional

or pooling layer, with kernel size k, we conduct
zero padding of the layer’s input with size ⌊k/2⌋._
来自Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
可能是我理解有误，还有这个.caffemodel下载不下来啊，需要翻墙吗？
谢谢大神

can`t open the link of the Spatial net model(v2) and Temporal net model(v2)

Click on the Spatial net model(v2) can`t download ,how i can download the two models? look forward to your answer!

Different features for the same video in matlab and VS2013.

Hello @wanglimin
I have download your modified iDT feature code and compiled it in the release mode by myself in VS2013 win10, then I have the .exe file .
I can not understand why Some videos in 'hmdb51_sta.' have different number of trajectories and features in Matlab using the command system(['./DenseTrackStab -f ',vid_name,' -o ',vid_name(1:end-4),'.bin']) compared with running the code in vs12013.

Thanks,
Mark