How can I create a matmul primitive with A16W8 (active 16bits, weight 8bits) configuration?

[Teaonly]

How can I create a matmul primitive with A16W8 (active 16bits, weight 8bits) configuration?

[Teaonly]

$ ./examples/tutorials-matmul-inference-int8-matmul-cpp gpu
onednn_verbose,info,oneDNN v3.6.0 (commit 95c00ed)
onednn_verbose,info,cpu,runtime:OpenMP,nthr:22
onednn_verbose,info,cpu,isa:Intel AVX2 with Intel DL Boost
onednn_verbose,info,gpu,runtime:OpenCL
onednn_verbose,info,gpu,engine,0,name:Intel(R) Arc(TM) Graphics,driver_version:24.9.28717,binary_kernels:enabled
onednn_verbose,primitive,info,template:operation,engine,primitive,implementation,prop_kind,memory_descriptors,attributes,auxiliary,problem_desc,exec_time
onednn_verbose,primitive,create:dispatch,gemm,gpu,gemm,jit:xe_hp:gemm:any,undef,src_a_f16::blocked:ab::f0 src_b_s8::blocked:ab::f0 dst_f16::blocked:ab::f0,attr-scales:wei:2:f32 attr-post-ops:eltwise_relu,,*x96:96x1000,skipping or dispatching to another implementation,src/gpu/intel/jit/gemm/xe_hp_systolic_gemm.cpp:75
onednn_verbose,primitive,create:dispatch,gemm,gpu,gemm,ocl:gemm_with_po:any,undef,src_a_f16::blocked:ab::f0 src_b_s8::blocked:ab::f0 dst_f16::blocked:ab::f0,attr-scales:wei:2:f32 attr-post-ops:eltwise_relu,,*x96:96x1000,runtime dimension is not supported,src/gpu/intel/ocl/gemm/gemm_with_post_ops.cpp:42
onednn_verbose,primitive,create:dispatch,gemm,gpu,gemm,jit:gemm:any,undef,src_a_f16::blocked:ab::f0 src_b_s8::blocked:ab::f0 dst_f16::blocked:ab::f0,attr-scales:wei:2:f32 attr-post-ops:eltwise_relu,,*x96:96x1000,unsupported datatype,src/gpu/intel/jit/gemm/gen_gemm.hpp:124
onednn_verbose,primitive,create:dispatch,gemm,gpu,gemm,ocl:ref:any,undef,src_a_f16::blocked:ab::f0 src_b_s8::blocked:ab::f0 dst_f16::blocked:ab::f0,attr-scales:wei:2:f32 attr-post-ops:eltwise_relu,,*x96:96x1000,unsupported attribute,src/gpu/intel/ocl/gemm/ref_gemm.hpp:81
onednn_verbose,primitive,create:dispatch,matmul,failed to create nested primitive gemm,src/gpu/intel/ocl/gemm_matmul.hpp:266
onednn_verbose,primitive,create:dispatch,matmul,gpu,matmul,ocl:ref:any,undef,src_f16::blocked:ab::f0 wei_s8::blocked:ab::f0 dst_f16::blocked:ab::f0,attr-scales:wei:2:f32 attr-post-ops:eltwise_relu,runtime_dims_masks:1:0,*x96:96x1000,unsupported datatype combination,src/gpu/intel/ocl/ref_matmul.hpp:70
oneDNN error caught:
Status: unimplemented
Message: could not create a primitive descriptor for a matmul primitive
Example failed on GPU.

[igorsafo]

Hi @Teaonly , here is an example: https://github.com/oneapi-src/oneDNN/blob/main/examples/tutorials/matmul/weights_decompression_matmul.cpp (or https://oneapi-src.github.io/oneDNN/page_weights_decompression_matmul_cpp.html#doxid-weights-decompression-matmul-cpp)
The fpmath_mode should be set to force int8 operation to work with floating point computations.

For more information please review a discussion on the same topic: #1893