[legacy-nn] losses need to return tensors and not numbers

[soumith]

[legacy-nn] losses need to return tensors and not numbers

[apaszke]

Right. In new nn the only ones to fix are
in THCUNN:

• MultiLabelMarginCriterion
• MultiMarginCriterion
• PReLU

and in Python:

• CosineEmbeddingLoss
• HingeEmbeddingLoss
• MarginRankingLoss

[apaszke]

As a side note, this is a case only in legacy nn. New nn is just fine 😊

[fmassa]

A number of criterions still use .sum (), which calls sumall and returns a number. The THC interface might need to be changed to return a 1d tensor instead.

[fmassa]

Maybe a better workaround for the moment would be to replace the .sum() by tensor.view(-1).sum(1) ? We don't yet have tensor broadcasting and returning a 1d tensor in sumall / meanall / etc might be anoying for simple data processing (like subtracting the mean and dividing by the std).

[apaszke]

No, we should definitely leave tensor sumall/meanall as they are right now, but we should expose non-synchronizing alternatives (e.g. for use in autograd, where we never return a scalar). Good idea with view + sum. It might break for non-contiguous inputs, but I guess we don't care about these cases very much (as most modules don't accept them anyway).

[soumith]

i dont think we should fix legacy-nn, i'm going to close this issue.