Why is the convolution performance of bf16 using opencl very low?

[feixuedudiao]

Why is the convolution performance of bf16 using opencl very low?

[yehudaorel]

Hi @feixuedudiao, Intel(R) UHD Graphics 630 does not have native hardware accelerator for bf16 operations. If I recall correctly, Gen9(.5) has native support for fp16 & fp32 operations. Given these hardware constraints, bf16 operations might be up/down converted behind the scenes to fp32, which can introduce overhead and impact performance. In addition, post-ops are typically optimized for specific data types, and the implementation of hardswish post-op might be more efficient for fp16 or fp32 compared to bf16 on this particular arch.

[feixuedudiao]

@yehudaorel .Thank you to answer my question, but i have a problem that which model of cpu can support the bf16 operations, and which can support fp16 opeartions. I have not find in the document.

[yehudaorel]

@feixuedudiao, you can refer to oneDNN docs for hardware limitations and supported Instruction Set Architectures (ISA's) for a specific data type. Please note that ISA's such as AVX512-FP16 and AMX-BF16 are only found on Intel Xeon Scalable Processor family. Similarly you can checkout optimized GPU microarchitecture for supported dtypes.

I would highly recommend to invest in a dedicated GPU if you plan on running compute intensive AI workloads. Intel(R) Arc(TM) Graphics is an example which supports native optimizations for [f32, f16, bf16, s8, u8] data types.

If you would rather go the client CPU & iGPU route I would recommend to checkout Intel(R) Core(TM) Ultra Processors (codename Meteor Lake) which documents the following dtype support:

CPU(Intel® Core™ Ultra 7 processor 155UL):

iGPU(Intel® Core™ Ultra 7 processor 155UL):

NPU(Intel® Core™ Ultra 7 processor 155UL):

Note that NPU engine is currently only supported via openVINO framework.

Hope this helps!