Why do the "reorder" operations of the same operator take very different times on the CPU and GPU platforms?

[feixuedudiao]

Why do the "reorder" operations of the same operator take very different times on the CPU and GPU platforms?

[vpirogov]

@feixuedudiao,

Performance differences are expected on different platforms. One thing to note though is that oneDNN verbose mode has non-trivial performance overhead, in particular on GPUs and cannot be reliably used to measure performance. You can use benchdnn in performance validation mode to get accurate performance measurements.

[feixuedudiao]

@vpirogov. I conducted time consumption tests based on the cpu and gpu for the reordering of convolutional src and weights respectively, the code is from example of primitime convolution.cpp. The result of cpu and gpu are respectively 986 microseconds and 999604 microseconds，gpu is many times slower than cpu. Is there a better way to improve the performance of gpu reoder?

[shu1chen]

@feixuedudiao First of all, it doesn't make sense to compare the performance of primitives on CPU and GPU without considering the GPU hardware capabilities and configurations.
For your case, if you insist on comparing them, please try performance testing mode of benchdnn for testing. Here is an example of command line to check the 32x32x1x1 reorder:

./benchdnn --reorder --mode=P --reset --allow-enum-tags-only=0 --engine=gpu  --runtime-dim-mask=  --sdt=f32 --ddt=f32  --stag=abcd --dtag=Acdb16a --strides=:   32x32x1x1

I tested this command line on a new laptop with a latest Intel integrated GPU hardware and it shows that the performance on GPU is better than that on CPU:
Avg. time on CPU: 0.00714332 ms
Avg. time on GPU: 0.00160363 ms