result from aimet evaluation and result after quantization on 8295 doesn't match

[superpigforever]

result from aimet evaluation and result after quantization on 8295 doesn't match

[quic-akinlawo]

Hello @superpigforever, batchnorms are optimized out during conversion by folding the encoding values into the preceding conv2d (including depthwise and transpose variants) or fully connected layers. As such, the missing batchnorm operation is expected.

I would recommend a layer wise comparison between the fp32 model and the QNN quantized model. That could help narrow down the source of the regression.

[e-said]

Hi @superpigforever,

There are two points I would recommend checking:

1/ BN folding during QAT (using the method fold_all_batch_norms) => this is recommended to ensure consistency between QAT and hardware inference.
2/ Ensure that the encodings in the cpp file generated by qnn-onnx-converter contains the encodings coming from aimet QAT.