Support for 1D and Non-square convolutional kernels

[hughperkins]

Support for 1D and Non-square convolutional kernels

[hughperkins]

Understood. Involves a fair amount of work. Will ponder. Anyone else who arrives in this thread, and wants non-square kernels, please add a +1 into this thread, to motivate me a bit :-)

[hughperkins]

Oh, hmmm, kernels.... is this for Inception?

[brazilla]

Mainly for NLP and other sequence processing.

[hughperkins]

Ah, hmmm, right.

Presumably you'd want the input 'images' to also be non-squared too? Presumably, quite very not-square, like 1-dimensional?

[brazilla]

Yeah,exactly.

[hughperkins]

Hmmm... does sound potentially quite useful generally. By the way, just out of curiosity do you know any papers that use this technique?

[brazilla]

http://nal.co/papers/Kalchbrenner_DCNN_ACL14
http://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf
http://www-nlp.stanford.edu/pubs/SocherLinNgManning_ICML2011.pdf

[hughperkins]

Thanks!

[wbernoudy]

+1. Has there been any progress with this?

[hughperkins]

Hmmmm .... well.... clnn contains a wrapper for 1-d convolutions. https://github.com/hughperkins/clnn/blob/master/TemporalConvolution2.lua I confess I'm unlikely to dedicate any time to implementing this in DeepCL in the short-term. I dont imagine it's tons of work, as long as you:

• make it only for 1-d, ie not generally rectangular support
• use the existing im2col convolution implementation. It's fast, and means you wont need to roll up your sleeves and dive into opencl too much (might need to ... a bit... for im2col ... maybe :-P)
  It is however non-trivial enough that I'm not just going to spend a few hours one morning, and make it work. It's probably about a weeks work, I would think, ballpark. 40-80 man-hours or so.

[hughperkins]

Note: I think an appropriate way forward for this might be to migrate to use BVLC/caffe#4155 , so that any kernels are shared across OpenCL projects.

[dex1990]

hi,@hughperkins
did DeepCL support 3D square convolutional layers ? e.g. 3x3x64, which means size of filter is width 3,height 3 and depth 64.

[hughperkins]

Correct. Can you confirm what makes you feel this might not be the case?

On 15 November 2016 10:35:01 GMT+00:00, dex1990 [email protected] wrote:

hi,@hughperkins
did DeepCL support 3D square convolutional layers ? e.g. 3x3x64, which
means size of filter is width 3,height 3 and depth 64.

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#37 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[dex1990]

Thank you for your answer.
I have not found any example on it, maybe i missed some relevant examples.
I have trained model from this project:
https://github.com/huangzehao/caffe-vdsr
some layers of this network have input size: 40x40x64 and 64 filters with size: 3x3x64, so the output size is 40x40x64. I want to test the trained network with DeepCL.

[0StackOverflow0]

I'm interested in adding 3D convolutional filters (once I do some more reading), but I'm hoping to have a proposal for stride in the not too distant future.

I wonder how hard 1D would be after doing that.

[hughperkins]

I'm hoping to have a proposal for stride in the not too distant future.

Sounds good :)

I wonder how hard 1D would be after doing that.

Its not conceptually hard, but it might involve a certain amount of legwork :)