Why does yolov3 run slower on Tesla T4 than on RTX2080ti？

[ceccocats]

Why does yolov3 run slower on Tesla T4 than on RTX2080ti？

[mrhosseini]

Actually RTX2080ti is faster than Tesla T4. See the below links:

https://www.servethehome.com/nvidia-tesla-t4-ai-inferencing-gpu-review/4/

https://versus.com/en/nvidia-geforce-rtx-2080-ti-founders-edition-vs-nvidia-tesla-t4

[rotorliu]

@mrhosseini Thank you
Why is it that T4 FLOPs is higher but slower？

[mrhosseini]

Why is it that T4 FLOPs is higher but slower？

On FP32 RTX2080ti is faster. On FP16 T4 has a higher FLOPS and should be faster. If not may be possible bug in tkDNN or TensorRT. May be not you are not using the latest version TensorRT.

[rotorliu]

Why is it that T4 FLOPs is higher but slower？

On FP32 RTX2080ti is faster. On FP16 T4 has a higher FLOPS and should be faster. If not may be possible bug in tkDNN or TensorRT. May be not you are not using the latest version TensorRT.

The table above is the result of FP16 mode. The TensorRT version is v7. So I am confused.
The code is used：
https://github.com/ceccocats/tkDNN/blob/master/tests/test_rtinference/rtinference.cpp

[rotorliu]

This test also shows that T4 is relatively slow.

[ceccocats]

Source (Page 60):
https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf&ved=2ahUKEwjwsLzNtZPqAhXC3KQKHUJfCLUQFjAGegQIARAB&usg=AOvVaw3CuG9m9TLCpXjybLqkvyJR&cshid=1592760127145

[rotorliu]

Thanks