PytorchDemoApp at 1.4_nightly

[pytorch]

PytorchDemoApp at 1.4_nightly

[ovthf]

Hello @IvanKobzarev

Thanks for your quick response.
The issue has been fixed.

I noticed another issue that quantized model was much slower than floating model on moto G7 power android phone.

With floating model, duration per inference is around 3 second.
With quantized model, duration per inference is around 10 second, and it outputs below message
"E/QNNPACK: failed to create convolution with 0.03779296 input scale, 2.583814 kernel scale, and 0.06601464 output scale: convolution scale 1.479217 is greater or equal to 1.0"

Please help to investigate.

Thanks a lot.

[zetyquickly]

Same here #31

[IvanKobzarev]

Hello @ovthf
Sorry for my late reply.

There was a bug that was introduced into nightly builds - they included debug symbols and were very big, that gradle without additional setup of heap size (Xmx) failed to transform them.

The fix was landed in master today ( pytorch/pytorch@8e3486d ) and the latest nightlies are 'normal' size.

Please try the latest nightlies if it is still the problem.

[ovthf]

Hello @IvanKobzarev

Thanks for your help.

It works fine now.

But when doing second time forward, encounter below issue. Please help. Thanks a lot.

Testing code:
// running the model
final Tensor outputTensor = module.forward(IValue.from(inputTensor)).toTensor(); // first time is OK
final Tensor outputTensor2 = module.forward(IValue.from(inputTensor)).toTensor();

com.facebook.jni.CppException:
Unknown builtin op: aten::_adaptive_avg_pool2d_backward.
Could not find any similar ops to aten::_adaptive_avg_pool2d_backward. This op may not exist or may not be currently supported in TorchScript.
:
at :9:28
grad_self = grad.expand(self.size()) / (self_size[-1] * self_size[-2])
else:
grad_self = torch._adaptive_avg_pool2d_backward(grad, self)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE

            return grad_self

    at org.pytorch.Module$NativePeer.forward(Native Method)

[IvanKobzarev]

@ovthf
Yes, its connected with the issue that was fixed in pytorch/pytorch#30067

It was merged yesterday in pytorch master and it is included in our nightly android builds. To use them:

repositories {
    maven {
        url "https://oss.sonatype.org/content/repositories/snapshots"
    }
}

dependencies {
    ...
    implementation 'org.pytorch:pytorch_android:1.4.0-SNAPSHOT'
    implementation 'org.pytorch:pytorch_android_torchvision:1.4.0-SNAPSHOT'
    ...
}

If you already using nightlies, you might need gradle key --refresh-dependencies to be sure that gradle would not use the cached version.

To check that the patch is included, 1.4.0 nightlies version should be >= 45
the latest nightly url:
https://oss.sonatype.org/service/local/repositories/snapshots/content/org/pytorch/pytorch_android/1.4.0-SNAPSHOT/pytorch_android-1.4.0-20191120.095738-45.aar

[dreiss]

@raghuramank100 , have you seen that error before?

[raghuramank100]

I havent seen this before, but this error is specific to QNNPACK and is related to allowed values of scale and zero-point. I havent seen this issue for Mobilenet-v2.

[raghuramank100]

This looks like a genuine error related to QNNPACK, it is coming from: https://github.com/pytorch/QNNPACK/blob/901e9d40aeedb2c1b212b28f022fb099ffe617c2/src/convolution.c#L161
There is an assumption that (input_scale*wt_scale) < output_scale.

Are you running your model with sufficient calibration data prior to quantizing it?
cc @AshkanAliabadi @supriyar

[ovthf]

Used 50 images to do calibration.

Here is how to quantize model:

model = torch.load(m_path)
torch.backends.quantized.engine = "qnnpack"
model.eval()
model.fuse_model()
model.qconfig = torch.quantization.get_default_qconfig("qnnpack")
torch.quantization.prepare(model, inplace=True)
calibrate_model(model)
torch.quantization.convert(model, inplace=True)
example = torch.rand(1, 1, width, height)
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save(out_file)

Please help to verify.