autocast for inference

[plemeri]

autocast for inference

[plemeri]

I think the autocast that you are trying to do is to use lower precision, such as half precision.
It is convenient but could throw some error like this.

I'm not sure about the result but you can try casting manually.
I think you can use a wrapper to change the model itself to support lower precision.

Also, I personally do not recommend using half precision for all layers. Use half precision on backbone network only instead.

Thanks.