Including sparse convolutions inside traditional model

[fabvio]

Including sparse convolutions inside traditional model

[btgraham]

Using sparse convolutions only makes sense if the input is spatially sparse. What is your input?

The output of dense convolutions will not be sparse, so you should not use dense convolutions followed by sparse convolutions.

[fabvio]

Hi, thanks for your reply.
My input is actually sparse. In my network, after some convolutional layers, I apply a binary mask to the pixels having a low probability to be classified correctly, and I propagate only that set of pixels to the deeper layers. I would like to apply the sparse convolution in those layers.

[btgraham]

That might work. But the learning signal is the dense layers will be limited to the sites that are not filtered out, which could be problematic.
Can you not use sparse filters end-to-end?

[fabvio]

I think that this should not be an issue, correct me if I'm wrong. I'll give you some more information about my network, so that you could better understand what I'm trying to do.
After the application of the binary mask to the input, the filtered pixels are not lost, but they are directly connected to a layer where I overlap the results of the sparse convolution (after the application of a SparseToDense module) and the dense convolutive layers. In this way, I could use the last part of my network (the sparse one) to learn difficult features in an efficient way, and the first part (the dense one) to learn both difficult and easy features. Joining the results of the two classification, I should be able to backpropagate the learning signals for both the sites with difficult and easy classification. What do you think about it?

[fabvio]

Could I simply implement a layer where I map my input to an InputBatch (in the updateOutput function), like you do in your example, and the sparse gradOutput to a dense gradInput (in the updateGradInput)?

[btgraham]

Please contact me at [email protected] so we can discuss this in more detail.
Regards
Ben