Loading pre-trained models, e.g. VGG

[joschu]

Loading pre-trained models, e.g. VGG

[benanne]

I think the general consensus is that local response normalization is sort of pointless. To the best of my knowledge people aren't really using this anymore. That said, if you want to reproduce existing models exactly I guess you have no choice but to implement it. It's just a bit unfortunate to be spending valuable development time on stuff like this :)

We have a pure-Theano implementation in Lasagne: https://github.com/Lasagne/Lasagne/blob/master/lasagne/layers/normalization.py#L47
That should translate to CGT fairly well, I assume! It's not the fastest, but maybe it'll save you some time.

[joschu]

That's interesting to hear that LRN is not considered to be important anymore, and thanks for the pointer to the Lasagne code.

I was also thinking about implementing the cross-channel LRN with a matrix multiplication (after permuting the axes so that channels are last) and the 2D neighborhood version as im2col + matrix multiplication.