Dropout in last layer

[weihua916]

Dropout in last layer

[weihua916]

It's just a regularization.

[foxtrotmike]

Thank you for your response - Tome, it makes sense to regularize using dropout at layers before the last one. Dropout at the prediction output forces prediction scores of an example to become zero and also scales up the prediction scores of other examples in the batch. Please see: "Furthermore, the outputs are scaled by a factor of \frac{1}{1-p}" in https://pytorch.org/docs/stable/generated/torch.nn.Dropout.html
It would be great if you could help me understand this a bit better. Thanks, again.