gpu+cuDNN slower than gpu only on my data and networks

[xjtuljy]

gpu+cuDNN slower than gpu only on my data and networks

[happynear]

Yes, I have also observed the phenomenon. That's why I have rolled back to cuDNN v2.
cuDNN v2 has also been unstable until rc3. So I tend to wait NVIDIA's engineers optimizing it for a few months.

[happynear]

Actually, cuDNN v3 have provided various algorithms for convolution. In layer setup step, convolution layer will automatically select one algorithm based on the GPU memory. However, this feature is not exploited well. You may try to force caffe using some specified algorithms for forward and backward, and test how much time they would cost.

I have already done some experiment on my GTX 780. Unfortunately, none combination of algorithms performs better than cuDNN v2. I guess cuDNN v3 is just made for Maxwell architecture GPUs. You may try by yourself. I am looking forward for your results to check if this is a windows problem or just due to the unmature cuDNN.

[xjtuljy]

Thanks for your reply. I just tried it on my GTX980 which is Maxwell architecture GPU, and found it much slower than gpu alone (windows 7). I already switched back to cuDNN v2 which is marginally faster than v1 with my data.

[happynear]

This problem has been solved in the most recent version of cudnn. I have updated the 3rdparty libarary package and the codes. Please check.