Is your proposal related to a problem?
The current version of EZKL does not support the verification of torch.nn.ConvTranspose2d and torch.nn.Upsample (corresponding to the DeconvUnary operation and Resize operation in the onnx format).
Pytorch Docs
torch.nn.ConvTranspose2d (https://pytorch.org/docs/stable/generated/torch.nn.ConvTranspose2d.html)
torch.nn.Upsample (https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html)
Test File and Result
torch.nn.ConvTranspose2d
Generate input.json and network.onnx:
import io
import numpy as np
from torch import nn
import torch.onnx
import torch.nn as nn
import torch.nn.init as init
import json
class Circuit(nn.Module):
def __init__(self, inplace=False):
super(Circuit, self).__init__()
self.convtranspose = nn.ConvTranspose2d(3, 3, (5, 5), stride=2, padding=2, output_padding=1)
def forward(self, x):
y = self.convtranspose(x)
return y
def main():
torch_model = Circuit()
# Input to the model
shape = [3, 5, 5]
x = 0.1*torch.rand(1,*shape, requires_grad=True)
torch_out = torch_model(x)
# Export the model
torch.onnx.export(torch_model, # model being run
(x), # model input (or a tuple for multiple inputs)
"network.onnx", # where to save the model (can be a file or file-like object)
export_params=True, # store the trained parameter weights inside the model file
opset_version=10, # the ONNX version to export the model to
do_constant_folding=True, # whether to execute constant folding for optimization
input_names = ['input'], # the model's input names
output_names = ['output'], # the model's output names
dynamic_axes={'input' : {0 : 'batch_size'}, # variable length axes
'output' : {0 : 'batch_size'}})
d = ((x).detach().numpy()).reshape([-1]).tolist()
data = dict(input_shapes = [shape],
input_data = [d],
output_data = [((o).detach().numpy()).reshape([-1]).tolist() for o in torch_out])
# Serialize data into file:
json.dump(data, open("input.json", 'w'))
if __name__ == "__main__":
main()
After setup by execute ezkl gen-srs --logrows 17 --params-path=kzg.params
, I try to verify the inference result of network by execute ezkl mock --tolerance 2 --scale 4 --bits 16 -K 17 -D ./input.json -M ./network.onnx
and get the following error information:
[*] [0s, ezkl] -
|
| ███████╗███████╗██╗ ██╗██╗
| ██╔════╝╚══███╔╝██║ ██╔╝██║
| █████╗ ███╔╝ █████╔╝ ██║
| ██╔══╝ ███╔╝ ██╔═██╗ ██║
| ███████╗███████╗██║ ██╗███████╗
| ╚══════╝╚══════╝╚═╝ ╚═╝╚══════╝
|
| -----------------------------------------------------------
| Easy Zero Knowledge Learning.
| -----------------------------------------------------------
|
|
[*] [0s, ezkl] - command:
| {
| "command": {
| "Mock": {
| "args": {
| "allocated_constraints": null,
| "bits": 16,
| "logrows": 17,
| "pack_base": 1,
| "public_inputs": false,
| "public_outputs": true,
| "public_params": false,
| "scale": 4,
| "tolerance": {
| "Abs": {
| "val": 2
| }
| }
| },
| "data": "./input.json",
| "model": "./network.onnx"
| }
| }
| }
[*] [0s, ezkl_lib::commands] - loading data from ./input.json
[W] [0s, ezkl_lib::graph::utilities] - Unknown op: DeconvUnary
[*] [0s, ezkl_lib::graph::model] - calculating num of constraints using dummy model layout...
[E] [0s, ezkl_lib::graph::model] - unsupported operation in graph
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Synthesis', src/graph/model.rs:223:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
torch.nn.Upsample
Similarly, I generate input.json and network.onnx through the following file:
import io
import numpy as np
from torch import nn
import torch.onnx
import torch.nn as nn
import torch.nn.init as init
import json
class Circuit(nn.Module):
def __init__(self, inplace=False):
super(Circuit, self).__init__()
self.upsample = nn.Upsample(scale_factor=2, mode='nearest')
def forward(self, x):
y = self.upsample(x)
return y
def main():
torch_model = Circuit()
# Input to the model
shape = [3, 5, 5]
x = 0.1*torch.rand(1,*shape, requires_grad=True)
torch_out = torch_model(x)
# Export the model
torch.onnx.export(torch_model, # model being run
(x), # model input (or a tuple for multiple inputs)
"network.onnx", # where to save the model (can be a file or file-like object)
export_params=True, # store the trained parameter weights inside the model file
opset_version=11, # the ONNX version to export the model to
do_constant_folding=True, # whether to execute constant folding for optimization
input_names = ['input'], # the model's input names
output_names = ['output'], # the model's output names
dynamic_axes={'input' : {0 : 'batch_size'}, # variable length axes
'output' : {0 : 'batch_size'}})
d = ((x).detach().numpy()).reshape([-1]).tolist()
data = dict(input_shapes = [shape],
input_data = [d],
output_data = [((o).detach().numpy()).reshape([-1]).tolist() for o in torch_out])
# Serialize data into file:
json.dump(data, open("input.json", 'w'))
if __name__ == "__main__":
main()
I get the error information as following:
[*] [0s, ezkl] -
|
| ███████╗███████╗██╗ ██╗██╗
| ██╔════╝╚══███╔╝██║ ██╔╝██║
| █████╗ ███╔╝ █████╔╝ ██║
| ██╔══╝ ███╔╝ ██╔═██╗ ██║
| ███████╗███████╗██║ ██╗███████╗
| ╚══════╝╚══════╝╚═╝ ╚═╝╚══════╝
|
| -----------------------------------------------------------
| Easy Zero Knowledge for Liberty.
| -----------------------------------------------------------
|
|
[*] [0s, ezkl] - command:
| {
| "command": {
| "Mock": {
| "args": {
| "allocated_constraints": null,
| "bits": 16,
| "logrows": 17,
| "pack_base": 1,
| "public_inputs": false,
| "public_outputs": true,
| "public_params": false,
| "scale": 4,
| "tolerance": {
| "Abs": {
| "val": 2
| }
| }
| },
| "data": "./input.json",
| "model": "./network.onnx"
| }
| }
| }
[*] [0s, ezkl_lib::commands] - loading data from ./input.json
[W] [0s, ezkl_lib::graph::utilities] - Unknown op: Resize
[*] [0s, ezkl_lib::graph::model] - calculating num of constraints using dummy model layout...
[E] [0s, ezkl_lib::graph::model] - unsupported operation in graph
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Synthesis', src/graph/model.rs:223:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Describe the solution you'd like
Support the verification of torch.nn.ConvTranspose2d and torch.nn.Upsample.
Possible Solution
It may be possible to realize the verification of torch.nn.ConvTranspose2d through the combination of padding verification and Conv verification.
Any insights are appreciated, thanks for the hard work!