alanlukezic/csr-dcf

Discriminative Correlation Filter with Channel and Spatial Reliability

csr-dcf's Issues

Failure Flag

Dear @alanlukezic,
Thank you for your nice work. I have question about your code.
How one can inform that your tracker fails to track the target?
I mean that, in some tracking algorithms we can inform when the tracker fails to track the target. For example, by using the L1-disnatce between the visual features of the target and the output of the tracker. More specifically, something like this:

if (L1-distance_between_target_bounding_box_and_tracker_output > threshold):
    print('The tracker fails to track the target')

I will really appreciate you if your answer cover your C++ implementation (i.e. both implementations (MATLAB & C++ one)).

Why the FPS is low when I run the demo_csr.m file?

I follow the instructions to compile the files and success to run the demo_csr.m on Inter Core i7-7700K 4.2GHz. However, the fps is so low, about 6. I have tried some other objects, the results range from 3 to 12 fps. The demo video shows that the tracking speed is high, can reach to 200 fps. Is it because of using gpu devices?

Issue for csr_wrapper

How to use the function of csr_wrapper to generate a output.txt file?

Issues for README

bounding box is not moving with the moving object

CSR-DCF has been incorporated in opencv and i have used in one of my object tracking project. However, at some point in tracking falling coins, the tracker's bounding box remains in a place for some time. This is undesirable. Is there any particular reason for this?

I have tried changing the psr_threshold as suggested here: https://stackoverflow.com/questions/54785230/csrt-algorithm-not-updating-target

However, is was able to make this box move, but again made another box stick to a spot for sometime.

How can I solve this issue. It'll be convenient if you can suggest some configurations of this algorithm via opencv.

Environment: Linux OS, python2.7
OpenCv: 4.1.1

Wrong type for dim?

csr-dcf/mex_src/hog/gradientMex.cpp

Line 336 in 469959d

const int *dims; int nDims;

argument of type "const int *" is incompatible with parameter of type "const mwSize={size_t={unsigned
__int64}} *"
mxSetData(M,*I); mxSetDimensions(M,dims,3); return M;

Should "dim" not rather be a "const unsigned __int64" (which is what mxSetDimensions and mxGetDimensions) expects?

Get psr_threshold without modifying the source code

Hello, is there a way to set the value of psr_threshold at the initialization of the CSRT Tracker ? I checked the public function TrackerCSRT::create(const TrackerCSRT::Params & parameters) and TrackerCSRT::Params::read(const FileNode &) but I wasn't able to set this parameter using a FileStorage. Any help/example would be appreciated.
Moreover, is there a public method to get the max_val from CSRTImpl::estimate_new_position in order to determine if the target is visible, partially lost or completely lost ?
PS : I don't want to modify the source code because I'm only using the module from openCV as it is.

Python version

Dear @alanlukezic,
Thank you for your fantastic project. Does any Python version of this project exists? (or any plan for release python version)

The results on OTB100

The results provided for otb100 has wrong with the sequence of clifbar, it has 472 frames, but the result only has 329 frames

about vot2017 results?

I run csr-dcf in vot2016 and its EAO is 0.3381, the same as claimed in paper. But the EAO in vot2017 is 0.2223, which is different from the EAO in the table of vot2017 results paper
http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w28/Kristan_The_Visual_Object_ICCV_2017_paper.pdf

Compile failure

Hello Mr.Alan Lukežič,
When i compiled the Compile.m function ,something wrong like this happened
Building with 'Microsoft Visual C++ 2015'.
Error using mex
gradientMex.cpp
C:\Users\Administrator\Desktop\csr-dcf-master\mex_src\hog\gradientMex.cpp(329):
error C2664: “int mxSetDimensions_730(mxArray *,const size_t *,std::size_t)”:
cannot convert from“const int [3]”to“const size_t *”
C:\Users\Administrator\Desktop\csr-dcf-master\mex_src\hog\gradientMex.cpp(329):
note: Irrelevant to the type of direction；Conversion requires reinterpret_cast, C style conversion, or function style conversion
C:\Users\Administrator\Desktop\csr-dcf-master\mex_src\hog\gradientMex.cpp(339):
warning C4267: “=”: From "size_t" to "int", may lose data
C:\Users\Administrator\Desktop\csr-dcf-master\mex_src\hog\gradientMex.cpp(339):
error C2440: “=”: cannot convert from“const size_t *”to“const int *”
C:\Users\Administrator\Desktop\csr-dcf-master\mex_src\hog\gradientMex.cpp(339):
note: Irrelevant to the type of direction；Conversion requires reinterpret_cast, C style conversion, or function style conversion

Error in compile (line 9)
mex gradientMex.cpp
the platform is MATLAB 2017b + VS2015+Opencv 2.4.13
so what should i do to fix it?
thank you in advance

Performance tips

Hello!
Thank you for your paper and code!
I'm planning to read the paper in the next days, but now I've tested OpenCV implementation of the algorithm and I can say it works a way better than other algos for my videos.
Can you please describe what params can I tune to increase fps rate but maybe sacrifice the precision. Also I have bg subtraction mask, maybe there is a way to utilize it inside the algorithm (with the code tweaks).
OpenCV params are listed here:
https://github.com/opencv/opencv_contrib/blob/6ef1983f0876fdf65083666d8e73abfecaf9d4f4/modules/tracking/src/trackerCSRT.cpp#L715

some wrong when run compile.m

gradientMex.cpp
D:\visual object tracking\Discriminative Correlation Filter with Channel and Spatial
Reliability\csr-dcf-master\mex_src\hog\gradientMex.cpp(329): error C2664: “int mxSetDimensions_730(mxArray *,const size_t
*,std::size_t)”: 无法将参数 2 从“const int [3]”转换为“const size_t *”
D:\visual object tracking\Discriminative Correlation Filter with Channel and Spatial
Reliability\csr-dcf-master\mex_src\hog\gradientMex.cpp(329): note: 与指向的类型无关；转换要求 reinterpret_cast、C 样式转换或函数样式转换
D:\visual object tracking\Discriminative Correlation Filter with Channel and Spatial
Reliability\csr-dcf-master\mex_src\hog\gradientMex.cpp(339): warning C4267: “=”: 从“size_t”转换到“int”，可能丢失数据
D:\visual object tracking\Discriminative Correlation Filter with Channel and Spatial
Reliability\csr-dcf-master\mex_src\hog\gradientMex.cpp(339): error C2440: “=”: 无法从“const size_t *”转换为“const int *”
D:\visual object tracking\Discriminative Correlation Filter with Channel and Spatial
Reliability\csr-dcf-master\mex_src\hog\gradientMex.cpp(339): note: 与指向的类型无关；转换要求 reinterpret_cast、C 样式转换或函数样式转换

出错 compile (line 9)
mex gradientMex.cpp

I don't know how to repair!!!

about your D3S

Hello Alan,
I'm Jesse , I read your paper D3S recently．　It is amazing.
Here are some confusions about your experiments in detail.
1:How to arrange 64 image pairs every batch to compute the foreground and background similarity channels?
2: In Videomatch , how the matching is perfomed in pytorch?
Looking forward for your letter.

Best wishes,
Jesse

Source code of CSR-DCF++

Do you have a plan to make the source code of CSR-DCF++ public available ?

What is channel?

Hello there,

I have a question regarding the meaning of "channel". I thought it meant the RGB channel, but after looking at the video of channel reliability weights visualization, there are so many of them. I thought there should be only 3 weights, one for each color channel.

If there are so many of them, then, channel can't be the RGB channel.

But by reading the code, I still think channel here is really the RGB channel, right?

Then how come the visualization shows so many weights?

Thanks,

mex error

/home/ubuntu/csr-dcf-master/mex_src/hog/gradientMex.cpp: In function ‘mxArray* mxCreateMatrix3(int, int, int,
mxClassID, bool, void**)’:
/home/ubuntu/csr-dcf-master/mex_src/hog/gradientMex.cpp:329:44: error: cannot convert ‘const int*’ to ‘const
mwSize* {aka const long unsigned int*}’ for argument ‘2’ to ‘int mxSetDimensions_730(mxArray*, const mwSize*, mwSize)’
mxSetData(M,I); mxSetDimensions(M,dims,3); return M;
^
/home/ubuntu/csr-dcf-master/mex_src/hog/gradientMex.cpp: In function ‘void checkArgs(int, mxArray*, int, const
mxArray**, int, int, int, int, int*, int*, int*, mxClassID, void**)’:
/home/ubuntu/csr-dcf-master/mex_src/hog/gradientMex.cpp:339:71: error: cannot convert ‘const mwSize* {aka const
long unsigned int*}’ to ‘const int*’ in assignment
nDims = mxGetNumberOfDimensions(pr[0]); dims = mxGetDimensions(pr[0]);
^

error compile (line 9)
mex gradientMex.cpp

How can I solve this problem？

How much speed up is expected if we use parallel programming like cuda?

Hi,
I'm curious about what happen if we take another step for this awesome algorithm and add GPU support to it? does the performance may be added dramatically ?

Tip to visualize spatial reliability

Hello, I want to visualize spatial reliability maps.
I try to do it with extract masks from the function tracker_csr_tracker and visualize it in visualize step.
What is exactly a spatial reliability map variable on your code?
I am confuse which I have to use among mask, valid_pixels_mask and fg.

I appreciate your answer.

Compilation Error

When I tried to compile on my Ubuntu machine, I get this error:

Error using mex
/home/ugurkart/Codebase/csr-dcf/mex_src/hog/gradientMex.cpp:
In function ‘mxArray* mxCreateMatrix3(int, int,
int, mxClassID, bool, void**)’:
/home/ugurkart/Codebase/csr-dcf/mex_src/hog/gradientMex.cpp:329:44:
error: cannot convert ‘const int*’ to ‘const
size_t* {aka const long unsigned int*}’ for
argument ‘2’ to ‘int mxSetDimensions(mxArray*,
const size_t*, size_t)’
mxSetData(M,I); mxSetDimensions(M,dims,3);
return M;
^
/home/ugurkart/Codebase/csr-dcf/mex_src/hog/gradientMex.cpp:
In function ‘void checkArgs(int, mxArray*,
int, const mxArray**, int, int, int, int, int*,
int*, int*, mxClassID, void**)’:
/home/ugurkart/Codebase/csr-dcf/mex_src/hog/gradientMex.cpp:339:48:
error: cannot convert ‘const size_t* {aka const
long unsigned int*}’ to ‘const int*’ in
assignment
nDims = mxGetNumberOfDimensions(pr[0]); dims
= mxGetDimensions(pr[0]);
^

Error in compile (line 9)
mex gradientMex.cpp

Any idea about how to solve it ?