Optimize the transformers

[facebookresearch]

Optimize the transformers

[fmassa]

Hi,

Thanks for the message.

Indeed, further research to make training for transformers (including DETR) faster is a very important research direction.
Another thing that should definitely be tried in order to increase accessibility is transfer learning, as discussed in #9

As for inference time, DETR is already pretty competitive with Faster R-CNN, as it can run at 28FPS on a V100 using a minimum image size of 800 (max image size 1333).
We believe that by using smaller backbones (for example MobileNetV2) and smaller transformers one could get an even faster inference runtime.

Thanks