jacenkow/inside

INSIDE: Steering Spatial Attention with Non-Imaging Information in CNNs

inside's Issues

`gamma` initialization

Hey @jacenkow, thanks again for the easy to use code!

I noticed from https://github.com/jacenkow/inside/blob/master/inside/layers.py#L56 that the affine transform is implemented as gamma * x + beta. As gamma is coming in from an FC layer initialized with defaults, it likely starts off very low and thus initially kills the input signal x.

It probably doesn't matter as SGD would just learn to undo it, but I typically see FiLM implemented as (1. + gamma) * x + beta. Based on very early qualitative testing I did with your code (1 + gamma)*x + beta seems to converge a bit faster than gamma*x + beta. Or maybe I'm just seeing what I want to see :)

Same applies for the INSIDE implementation as well.

w

Why there is no real code？