Should non-trivial irreps support bias or not?

[quva-lab]

Should non-trivial irreps support bias or not?

[prclibo]

Hi @Gabri95

Thanks for the reply! This explanation makes sense to me. It is fine to use nonlinearity instead.

Bo

[Gabri95]

Hi @prclibo

Thanks for your question!

indeed, the bias can only be added to the trivial representations.
One can show that adding a bias to any other irreducible representation would break equivariance.

However, what you suggest is not the usual notion of bias.
Usually, a linear/conv layer performs an affine transformation, while the bias applied to the norm is a non-linear operation.
For this reason, we chose to not include it in the convolution layer.
As we only needed this kind of bias in the implementation of HNet with norm-nonlinearities we chose to add this bias directly in the implementation of the non-linearity, see https://quva-lab.github.io/e2cnn/api/e2cnn.nn.html#e2cnn.nn.NormNonLinearity

If you need to this kind of bias outside the non-linearity, I can add it as a separate module

Gabriele