zkonduit/ezkl

ezkl is an engine for doing inference for deep learning models and other computational graphs in a zk-snark (ZKML). Use it from Python, Javascript, or the command line.

💭

EZKL

Easy Zero-Knowledge Inference

ezkl is a library and command-line tool for doing inference for deep learning models and other computational graphs in a zk-snark (ZKML). It enables the following workflow:

1. Define a computational graph, for instance a neural network (but really any arbitrary set of operations), as you would normally in pytorch or tensorflow.
2. Export the final graph of operations as an .onnx file and some sample inputs to a .json file.
3. Point ezkl to the .onnx and .json files to generate a ZK-SNARK circuit with which you can prove statements such as:

"I ran this publicly available neural network on some private data and it produced this output"

"I ran my private neural network on some public data and it produced this output"

"I correctly ran this publicly available neural network on some public data and it produced this output"

In the backend we use the collaboratively-developed Halo2 as a proof system.

The generated proofs can then be verified with much less computational resources, including on-chain (with the Ethereum Virtual Machine), in a browser, or on a device.

• If you have any questions, we'd love for you to open up a discussion topic in Discussions. Alternatively, you can join the ✨EZKL Community Telegram Group💫.

• For more technical writeups and details check out our blog.

• To see what you can build with ezkl, check out cryptoidol.tech where ezkl is used to create an AI that judges your singing ... forever.

---

getting started ⚙️

The easiest way to get started is to try out a notebook.

Python

Install the python bindings by calling.

pip install ezkl

Or for the GPU:

pip install ezkl-gpu

Google Colab Example to learn how you can train a neural net and deploy an inference verifier onchain for use in other smart contracts.

More notebook tutorials can be found within examples/notebooks.

CLI

Install the CLI

curl https://raw.githubusercontent.com/zkonduit/ezkl/main/install_ezkl_cli.sh | bash

ezklxdemo.mp4

For more details visit the docs. The CLI is faster than Python, as it has less overhead. For even more speed and convenience, check out the remote proving service, which feels like the CLI but is backed by a tuned cluster.

Build the auto-generated rust documentation and open the docs in your browser locally. cargo doc --open

In-browser EVM verifier

As an alternative to running the native Halo2 verifier as a WASM binding in the browser, you can use the in-browser EVM verifier. The source code of which you can find in the in-browser-evm-verifier directory and a README with instructions on how to use it.

building the project 🔨

Rust CLI

You can install the library from source

cargo install --locked --path .

ezkl now auto-manages solc installation for you.

building python bindings

Python bindings exists and can be built using maturin. You will need rust and cargo to be installed.

python -m venv .env
source .env/bin/activate
pip install -r requirements.txt
maturin develop --release --features python-bindings
# dependencies specific to tutorials
pip install torch pandas numpy seaborn jupyter onnx kaggle py-solc-x web3 librosa tensorflow keras tf2onnx

GPU Acceleration

If you have access to NVIDIA GPUs, you can enable acceleration by building with the feature icicle and setting the following environment variable:

export ENABLE_ICICLE_GPU=true

GPU acceleration is provided by Icicle

To go back to running with CPU, the previous environment variable must be unset instead of being switch to a value of false:

unset ENABLE_ICICLE_GPU

NOTE: Even with the above environment variable set, icicle is disabled for circuits where k <= 8. To change the value of k where icicle is enabled, you can set the environment variable ICICLE_SMALL_K.

contributing 🌎

If you're interested in contributing and are unsure where to start, reach out to one of the maintainers:

• dante (alexander-camuto)
• jason (jasonmorton)

More broadly:

• See currently open issues for ideas on how to contribute.

• For PRs we use the conventional commits naming convention.

• To report bugs or request new features create a new issue within Issues to inform the greater community.

Any contribution intentionally submitted for inclusion in the work by you shall be licensed to Zkonduit Inc. under the terms and conditions specified in the CLA, which you agree to by intentionally submitting a contribution. In particular, you have the right to submit the contribution and we can distribute it, among other terms and conditions.

no security guarantees

Ezkl is unaudited, beta software undergoing rapid development. There may be bugs. No guarantees of security are made and it should not be relied on in production.

NOTE: Because operations are quantized when they are converted from an onnx file to a zk-circuit, outputs in python and ezkl may differ slightly.

no warranty

Copyright (c) 2024 Zkonduit Inc. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

ezkl's Issues

The verification of torch.nn.ConvTranspose2d and torch.nn.Upsample

Is your proposal related to a problem?

The current version of EZKL does not support the verification of torch.nn.ConvTranspose2d and torch.nn.Upsample (corresponding to the DeconvUnary operation and Resize operation in the onnx format).

Pytorch Docs

torch.nn.ConvTranspose2d (https://pytorch.org/docs/stable/generated/torch.nn.ConvTranspose2d.html)
torch.nn.Upsample (https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html)

Test File and Result

torch.nn.ConvTranspose2d

Generate input.json and network.onnx:

import io
import numpy as np
from torch import nn
import torch.onnx
import torch.nn as nn
import torch.nn.init as init
import json

class Circuit(nn.Module):
    def __init__(self, inplace=False):
        super(Circuit, self).__init__()
        self.convtranspose = nn.ConvTranspose2d(3, 3, (5, 5), stride=2, padding=2, output_padding=1)

    def forward(self, x):
        y = self.convtranspose(x)
        return y

def main():
    torch_model = Circuit()
    # Input to the model
    shape = [3, 5, 5]
    x = 0.1*torch.rand(1,*shape, requires_grad=True)

    torch_out = torch_model(x)
    # Export the model
    torch.onnx.export(torch_model,               # model being run
                      (x),                   # model input (or a tuple for multiple inputs)
                      "network.onnx",            # where to save the model (can be a file or file-like object)
                      export_params=True,        # store the trained parameter weights inside the model file
                      opset_version=10,          # the ONNX version to export the model to
                      do_constant_folding=True,  # whether to execute constant folding for optimization
                      input_names = ['input'],   # the model's input names
                      output_names = ['output'], # the model's output names
                      dynamic_axes={'input' : {0 : 'batch_size'},    # variable length axes
                                    'output' : {0 : 'batch_size'}})

    d = ((x).detach().numpy()).reshape([-1]).tolist()

    data = dict(input_shapes = [shape],
                input_data = [d],
                output_data = [((o).detach().numpy()).reshape([-1]).tolist() for o in torch_out])

    # Serialize data into file:
    json.dump(data, open("input.json", 'w'))

if __name__ == "__main__":
    main()

After setup by execute ezkl gen-srs --logrows 17 --params-path=kzg.params, I try to verify the inference result of network by execute ezkl mock --tolerance 2 --scale 4 --bits 16 -K 17 -D ./input.json -M ./network.onnx and get the following error information:

[*] [0s, ezkl] - 
 |  
 |          ███████╗███████╗██╗  ██╗██╗
 |          ██╔════╝╚══███╔╝██║ ██╔╝██║
 |          █████╗    ███╔╝ █████╔╝ ██║
 |          ██╔══╝   ███╔╝  ██╔═██╗ ██║
 |          ███████╗███████╗██║  ██╗███████╗
 |          ╚══════╝╚══════╝╚═╝  ╚═╝╚══════╝
 |  
 |          -----------------------------------------------------------
 |          Easy Zero Knowledge Learning.
 |          -----------------------------------------------------------
 |          
 |          
[*] [0s, ezkl] - command: 
 |   {
 |    "command": {
 |      "Mock": {
 |        "args": {
 |          "allocated_constraints": null,
 |          "bits": 16,
 |          "logrows": 17,
 |          "pack_base": 1,
 |          "public_inputs": false,
 |          "public_outputs": true,
 |          "public_params": false,
 |          "scale": 4,
 |          "tolerance": {
 |            "Abs": {
 |              "val": 2
 |            }
 |          }
 |        },
 |        "data": "./input.json",
 |        "model": "./network.onnx"
 |      }
 |    }
 |  }
[*] [0s, ezkl_lib::commands] - loading data from ./input.json
[W] [0s, ezkl_lib::graph::utilities] - Unknown op: DeconvUnary
[*] [0s, ezkl_lib::graph::model] - calculating num of constraints using dummy model layout...
[E] [0s, ezkl_lib::graph::model] - unsupported operation in graph
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Synthesis', src/graph/model.rs:223:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

torch.nn.Upsample

Similarly, I generate input.json and network.onnx through the following file:

import io
import numpy as np
from torch import nn
import torch.onnx
import torch.nn as nn
import torch.nn.init as init
import json

class Circuit(nn.Module):
    def __init__(self, inplace=False):
        super(Circuit, self).__init__()
        self.upsample = nn.Upsample(scale_factor=2, mode='nearest')

    def forward(self, x):
        y = self.upsample(x)
        return y

def main():
    torch_model = Circuit()
    # Input to the model
    shape = [3, 5, 5]
    x = 0.1*torch.rand(1,*shape, requires_grad=True)

    torch_out = torch_model(x)
    # Export the model
    torch.onnx.export(torch_model,               # model being run
                      (x),                   # model input (or a tuple for multiple inputs)
                      "network.onnx",            # where to save the model (can be a file or file-like object)
                      export_params=True,        # store the trained parameter weights inside the model file
                      opset_version=11,          # the ONNX version to export the model to
                      do_constant_folding=True,  # whether to execute constant folding for optimization
                      input_names = ['input'],   # the model's input names
                      output_names = ['output'], # the model's output names
                      dynamic_axes={'input' : {0 : 'batch_size'},    # variable length axes
                                    'output' : {0 : 'batch_size'}})

    d = ((x).detach().numpy()).reshape([-1]).tolist()


    data = dict(input_shapes = [shape],
                input_data = [d],
                output_data = [((o).detach().numpy()).reshape([-1]).tolist() for o in torch_out])

    # Serialize data into file:
    json.dump(data, open("input.json", 'w'))

if __name__ == "__main__":
    main()

I get the error information as following:

[*] [0s, ezkl] - 
 |  
 |          ███████╗███████╗██╗  ██╗██╗
 |          ██╔════╝╚══███╔╝██║ ██╔╝██║
 |          █████╗    ███╔╝ █████╔╝ ██║
 |          ██╔══╝   ███╔╝  ██╔═██╗ ██║
 |          ███████╗███████╗██║  ██╗███████╗
 |          ╚══════╝╚══════╝╚═╝  ╚═╝╚══════╝
 |  
 |          -----------------------------------------------------------
 |          Easy Zero Knowledge for Liberty.
 |          -----------------------------------------------------------
 |          
 |          
[*] [0s, ezkl] - command: 
 |   {
 |    "command": {
 |      "Mock": {
 |        "args": {
 |          "allocated_constraints": null,
 |          "bits": 16,
 |          "logrows": 17,
 |          "pack_base": 1,
 |          "public_inputs": false,
 |          "public_outputs": true,
 |          "public_params": false,
 |          "scale": 4,
 |          "tolerance": {
 |            "Abs": {
 |              "val": 2
 |            }
 |          }
 |        },
 |        "data": "./input.json",
 |        "model": "./network.onnx"
 |      }
 |    }
 |  }
[*] [0s, ezkl_lib::commands] - loading data from ./input.json
[W] [0s, ezkl_lib::graph::utilities] - Unknown op: Resize
[*] [0s, ezkl_lib::graph::model] - calculating num of constraints using dummy model layout...
[E] [0s, ezkl_lib::graph::model] - unsupported operation in graph
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Synthesis', src/graph/model.rs:223:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Describe the solution you'd like

Support the verification of torch.nn.ConvTranspose2d and torch.nn.Upsample.

Possible Solution

It may be possible to realize the verification of torch.nn.ConvTranspose2d through the combination of padding verification and Conv verification.

Any insights are appreciated, thanks for the hard work!

Example leveraging the wasm prover / verifier

We currently have wasm-32-unknown-unknown verifiers and provers compiling and testing in headless chrome. Though (currently) undocumented it would be great to build toy apps / a tutorial using these.

If you need instructions on how to compile, fist install wasm-pack then run:

rustup target add wasm32-unknown-unknown
cargo install wasm-server-runner
# note you may need to add rust-src for your platform, for instance for linux: 
rustup component add rust-src --toolchain nightly-2022-11-03-x86_64-unknown-linux-gnu
# if you don't believe me that these things run correctly locally 
 wasm-pack test --chrome --headless -- -Z build-std="panic_abort,std"

The tutorial should prove / verify proofs directly in the browser

Create Releases with Compiled binaries and wasm

Describe the solution you'd like

There are no compiled binaries at the moment or releases made for the ezkl repo. It would be faster if people can download compiled binaries from the repo for immediate use. Compilation can take a while and require a user to properly setup rust.

create-wasm-verifier

Much like create-evm-verifier, we can have ezkl output a wasm verifier using the guidance in the halo2 book wasm section and zordle.

Flag for setting max number of used rows

Max number of "used" rows is currently hardcoded to 512. We should make this value settable using a cli flag / config field (possibly as part of .json config file update).

Random Forest / Decision Tree tensor op

Good news ! Sklearn models can be exported to ONNX ! Unfortunately they are often represented as single node within the ONNX graph. For instance in the case of a random forest:

We should try (if possible) to implement decision trees and random forests as a single operation within src/tensor/ops.rs, such that we can easily integrate such nodes into src/circuit/polynomial.rs (or maybe even src/circuit/lookup.rs if creating tree-based algos over field elements proves to be too challenging).

cargo build error (cannot locate remote-tracking branch 'origin/ac/send-sync-region')

Describe the bug

When I try to execute cargo build --release on the current version (bc5abf5), the following error occurs:

warning: unused manifest key: target.wasm32-unknown-unknown.runner
    Blocking waiting for file lock on package cache
    Updating git repository `https://github.com/zkonduit/halo2`
error: failed to get `halo2_proofs` as a dependency of package `ezkl-lib v0.1.0`

Caused by:
  failed to load source for dependency `halo2_proofs`

Caused by:
  Unable to update https://github.com/zkonduit/halo2?branch=ac/send-sync-region

Caused by:
  failed to find branch `ac/send-sync-region`

Caused by:
  cannot locate remote-tracking branch 'origin/ac/send-sync-region'; class=Reference (4); code=NotFound (-3)

I have checked https://github.com/zkonduit/halo2 and found that it has only one "main" branch, and there is no "ac/send-sync-region" branch.

I'm not sure whether the error occurred because of a wrong configuration of my environment or a wrong operation, so I'm looking for your help.

Any insights are appreciated, thanks for the hard work!

Fully dynamic layers

• Affine layers now pull their dimensionality from Val/VarTensors. We should apply this as much as possible to Envwise operations and Convolutional layers (some parameters like a convolutional layer's Stride and Padding can't be extracted from inputs).

compiler errors on the prototype

Hi, trying to learn this package , during assembly with rust toolchain nightly-2021-11-29-x86_64-apple-darwin) gives (below compiler output)

using halo2_proofs-0.2.0 in Cargo

error[E0599]: the method flatten exists for array [[T; HEIGHT]; WIDTH], but its trait bounds were not satisfied
--> src/cnvrl_generic/util.rs:94:10
|
94 | .flatten()
| ^^^^^^^ method cannot be called on [[T; HEIGHT]; WIDTH] due to unsatisfied trait bounds
|
= note: the following trait bounds were not satisfied:
[[T; HEIGHT]; WIDTH]: Iterator
which is required by &mut [[T; HEIGHT]; WIDTH]: Iterator
[[T; HEIGHT]]: Iterator
which is required by &mut [[T; HEIGHT]]: Iterator

error[E0599]: the method flatten exists for array [[T; HEIGHT]; WIDTH], but its trait bounds were not satisfied
--> src/cnvrl_generic/util.rs:96:27
|
96 | .zip(image_subset.flatten().iter())
| ^^^^^^^ method cannot be called on [[T; HEIGHT]; WIDTH] due to unsatisfied trait bounds
|
= note: the following trait bounds were not satisfied:
[[T; HEIGHT]; WIDTH]: Iterator
which is required by &mut [[T; HEIGHT]; WIDTH]: Iterator
[[T; HEIGHT]]: Iterator
which is required by &mut [[T; HEIGHT]]: Iterator

error[E0599]: the method flatten exists for array [[halo2_proofs::plonk::Expression<F>; _]; _], but its trait bounds were not satisfied
--> src/cnvrl_generic.rs:117:18
|
117 | .flatten()
| ^^^^^^^ method cannot be called on [[halo2_proofs::plonk::Expression<F>; _]; _] due to unsatisfied trait bounds
|
= note: the following trait bounds were not satisfied:
[[halo2_proofs::plonk::Expression<F>; _]; _]: Iterator
which is required by &mut [[halo2_proofs::plonk::Expression<F>; _]; _]: Iterator
[[halo2_proofs::plonk::Expression<F>; _]]: Iterator
which is required by &mut [[halo2_proofs::plonk::Expression<F>; _]]: Iterator

error[E0061]: this function takes 1 argument but 2 arguments were supplied
--> src/eltwise.rs:81:24
|
81 | let _ = cs.lookup("lk", |cs| {
| ________^^^^^^-----
| | |
| | expected 1 argument
82 | | let qlookup = cs.query_selector(qlookup);
83 | | vec![
84 | | (
... |
92 | | ]
93 | | });
| |- supplied 2 arguments
|
note: associated function defined here
--> //rust/src/halo2/halo2_proofs/src/plonk/circuit.rs:1009:12
|
1009 | pub fn lookup(
| ^^^^^^

InstanceNorm support

"InstanceNorm" is not currently supported and prompts an node error.

It's similar to batch norm but slightly different—mean and variance are calculated at inference time on the input.

batch norm: https://github.com/onnx/onnx/blob/main/docs/Operators.md#BatchNormalization

instance norm: https://github.com/onnx/onnx/blob/main/docs/Operators.md#InstanceNormalization

Flexibility for public inputs

• Currently only the model output can serve as a public input to the circuit.
• In the interest of providing a maximum amount of flexibility for users we want to be able to allow for other configurations (public data, public model etc...).
• This could be done via a few fields in the data.json file that would toggle which parts of the model / data are public and private.

unnecessary qualification on `cargo build --release`

Huggingface integration

This maybe a very big stretch goal given current model size and ops limitations, but inspirationally worth considering.
Could be a big unlock of applications on ezkl, if it is simple enough for app devs to to plug ezkl into the huggingface transformer APIs and demo spaces.

Cleanup integration tests

integration tests are currently really messy, and aren't standardized in such a way to enable extending them easy and systematic.

we need to:

• define a set of recurring test files across all ezkl commands
• automatically iterate across these files and insert them into the test functions

adding new tests is as simple as either extending the list of files we iterate over, or defining a new test command.

MaxPool tracking issue

• Naive max pool implementation (range check for each cell of each window)
• More optimized but still exact implementation
• Approximate but highly optimized implementation

`Result::unwrap() on an 'Err' value: OddLength'` Error when running `create-evm-verifier`

Here are the steps I followed.

1. I created a gen.py to generate a onnx file with pyezkl and generated the inputs.json and network.onnx file accordingly

import io
import numpy as np
from torch import nn
import torch.onnx
import torch.nn as nn
import torch.nn.init as init
import json
from ezkl import export

class Model(nn.Module):
    def __init__(self, inplace=False):
        super(Model, self).__init__()

        self.aff1 = nn.Linear(3,1)
        self.relu = nn.ReLU()
        self._initialize_weights()

    def forward(self, x):
        x =  self.aff1(x)
        x =  self.relu(x)
        return (x)


    def _initialize_weights(self):
        init.orthogonal_(self.aff1.weight)

circuit = Model()
export(circuit, input_shape = [3])

2. Next I generated the kzg.params as follows

ezkl -K=19 gen-srs --pfsys=kzg --params-path=kzg.params

3. After which I ran the following to generate the proof

ezkl --bits=16 -K=17 --public-inputs --public-outputs prove \
-D ./input.json \
-M ./network.onnx \
--proof-path proof.pf \
--vk-path proof.vk \
--params-path=kzg.params \
--transcript=evm

4. Then I ran the following to generate the evm verifier

ezkl -K=17 --bits=16 create-evm-verifier \
-D ./input.json \
-M ./network.onnx \
--pfsys=kzg \
--deployment-code-path proof.code \
--params-path=kzg.params \
--vk-path proof.vk \
--sol-code-path verify.sol

This led to the following error

[*] 
 |          ███████╗███████╗██╗  ██╗██╗
 |          ██╔════╝╚══███╔╝██║ ██╔╝██║
 |          █████╗    ███╔╝ █████╔╝ ██║
 |          ██╔══╝   ███╔╝  ██╔═██╗ ██║
 |          ███████╗███████╗██║  ██╗███████╗
 |          ╚══════╝╚══════╝╚═╝  ╚═╝╚══════╝
 |  
 |          -----------------------------------------------------------
 |          Easy Zero Knowledge for Liberty.
 |          -----------------------------------------------------------
 |          
[*] {"command":{"CreateEVMVerifier":{"data":"./input.json","model":"./network.onnx","params_path":"kzg.params","vk_path":"proof.vk","deployment_code_path":"proof.code","sol_code_path":"verify.sol","pfsys":"KZG"}},"args":{"tolerance":0,"scale":7,"bits":16,"logrows":17,"public_inputs":false,"public_outputs":true,"public_params":false,"max_rotations":512}}
[*] loading data from ./input.json
[*] visibility: (inputs: private, params: private, outputs: public)
[*] ------ scaled const node 2: 7 -> 14
[*] assigning configuration buckets to operations
[*] public inputs lengths: [1]
[*] loading params from "kzg.params"
[*] loading verification key from "proof.vk"
[*] visibility: (inputs: private, params: private, outputs: public)
[*] ------ scaled const node 2: 7 -> 14
[*] assigning configuration buckets to operations
[*] row cap: 3
[*] number of advices used: 5
[*] number of fixed used: 0
[*] number of instances used: 1
[*] configuring model
[*] output_shapes [[1]]
Warning: Yul is still experimental. Please use the output with care.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: OddLength', /Users/myuser/.cargo/git/checkouts/snark-verifier-972b60abf3a1213b/df03d89/snark-verifier/src/loader/evm/util.rs:124:25
stack backtrace:
   0:        0x102a7fbc6 - std::backtrace_rs::backtrace::libunwind::trace::h4f0e9c01f163c4f1
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:        0x102a7fbc6 - std::backtrace_rs::backtrace::trace_unsynchronized::hb39a678382d45ac4
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:        0x102a7fbc6 - std::sys_common::backtrace::_print_fmt::hbd5777a2fb61dee1
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/sys_common/backtrace.rs:65:5
   3:        0x102a7fbc6 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::he1f89b1287f05728
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/sys_common/backtrace.rs:44:22
   4:        0x102aa093a - core::fmt::write::hb9a1d6cab1bed673
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/core/src/fmt/mod.rs:1232:17
   5:        0x102a7853c - std::io::Write::write_fmt::h0949db1b5c61fce6
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/io/mod.rs:1684:15
   6:        0x102a7f9aa - std::sys_common::backtrace::_print::hb76c16138015f78e
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/sys_common/backtrace.rs:47:5
   7:        0x102a7f9aa - std::sys_common::backtrace::print::h8e468349d7519f72
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/sys_common/backtrace.rs:34:9
   8:        0x102a812d3 - std::panicking::default_hook::{{closure}}::h82b6603ca5d1b345
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:271:22
   9:        0x102a81028 - std::panicking::default_hook::h5830f012475db027
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:290:9
  10:        0x102a8190b - std::panicking::rust_panic_with_hook::hd67e8d898f426d5c
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:692:13
  11:        0x102a81824 - std::panicking::begin_panic_handler::{{closure}}::h6a63947e5e6a54b8
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:583:13
  12:        0x102a80009 - std::sys_common::backtrace::__rust_end_short_backtrace::h36012dee40fe1f28
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/sys_common/backtrace.rs:137:18
  13:        0x102a814ed - rust_begin_unwind
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:579:5
  14:        0x102a9dfe3 - core::panicking::panic_fmt::h2cc1687c896e545c
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/core/src/panicking.rs:64:14
  15:        0x102a9e4e5 - core::result::unwrap_failed::h715c60f28b381d9b
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/core/src/result.rs:1750:5
  16:        0x102922fb1 - snark_verifier::loader::evm::util::compile_yul::hf7f90817fb06fcf2
  17:        0x101a3b46d - ezkl::pfsys::evm::single::gen_evm_verifier::h338e05b48632184e
  18:        0x1017bbb01 - ezkl::execute::run::{{closure}}::h8ddeebef33d33c88
  19:        0x1017c3886 - tokio::runtime::park::CachedParkThread::block_on::h560b680d7d1b776d
  20:        0x10192f570 - tokio::runtime::scheduler::multi_thread::MultiThread::block_on::hfb6352390f149a45
  21:        0x101757d6a - ezkl::main::h3dfd80b7011631a5
  22:        0x101915c96 - std::sys_common::backtrace::__rust_begin_short_backtrace::ha81372c28a0deba6
  23:        0x1018c17dc - std::rt::lang_start::{{closure}}::hc0596ad399d4c743
  24:        0x102a72280 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h2f964c0cf434021c
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/core/src/ops/function.rs:287:13
  25:        0x102a72280 - std::panicking::try::do_call::h6ab21d4a277d6bdf
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:487:40
  26:        0x102a72280 - std::panicking::try::hfb01fd452e665622
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:451:19
  27:        0x102a72280 - std::panic::catch_unwind::h62c2de4aaf1cecf6
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panic.rs:140:14
  28:        0x102a72280 - std::rt::lang_start_internal::{{closure}}::h8215e427c091b029
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/rt.rs:148:48
  29:        0x102a72280 - std::panicking::try::do_call::h894c678608cb6b0f
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:487:40
  30:        0x102a72280 - std::panicking::try::hff882471419c3f31
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panicking.rs:451:19
  31:        0x102a72280 - std::panic::catch_unwind::hc57428bace4d90b6
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/panic.rs:140:14
  32:        0x102a72280 - std::rt::lang_start_internal::h10b5dd8fa4967274
                               at /rustc/c5c7d2b37780dac1092e75f12ab97dd56c30861d/library/std/src/rt.rs:148:20
  33:        0x101757f0c - _main

Add Halo2 matrix printer command

as described in the halo2 book

Refactor nonlinearity layout forward pass

The code starting here is confusing.

Docs Fix: Example Models Link

Hoping this is the right procedure. Checked out the repos and couldn't find one for the retype docs.

Summary

The link to ONNX examples in the retype docs is broken, and it looked to me like the original repo it pointed to is gone and not integrated elsewhere (I could be wrong). ONNX Model Zoo could be a good candidate for replacement.

Link Section of the Documentation

Command Line Interface - python and cli tutorial 🐍: "For samples of onnx files see here."

Dynamic Lookup Tables

Some operations, such as softmax, require variable-length lookup tables that are established at proving time. PSE's Halo2 fork already has support for dynamic lookup tables (lookup_any) which enable us to use Instance or Advice columns to use as lookup tables. We should incorporate this to provide an option for a dynamic lookup tables for current ops and future more complex ones.

Rescale and act layers

• Inputs to activation layers can be OOR for lookup tables.
• Need some sort of rescaling / calibration / quantization pass to prevent this.

Faster build setups for local dev

Currently build times take forever and its making iterating quickly on a local setup kind of a pain. In particular ethers related packages take forever (I'm looking at you ethers-contracts and ethers-middleware).

Describe the solution you'd like

Would love to either make the ethers compiles a (default) flag that is normally switched on for UX purposes (lots of users are developing specifically for the EVM so we don't want to get in their way); but that can be easily switched off if I'm deving locally.

Any other solutions that accelerate build time would be welcome.

Fuzz Testing for EZKL using Criterion

Is your proposal related to a problem?

Right now the tests on EZKL are limited to unit tests and specific tests. This can be risky suppose if EZKL were to be used for DeFi and more mission critical applications. As a result, there could be bad edge cases that could be very disruptive or damaging.

Describe the solution you'd like

Repurpose the benchmark testing library Criterion for fuzzing. Fuzz commonly used models and models used in the wilds for any edge cases.

Add decimal places to proving time

With recent improvements and calibration for low resource usage, most of our examples have a proof time that rounds to 0s.

Describe the solution you'd like

Add a decimal place or two to the proof time.

Update readme for ONNX

• Include instructions for usage with ONNX (conditional compile flag + example usage).

Community Documentation

Documentation for ezkl with community project examples. Should be:

• A separately existing site (not Rust docs) like a Gitbook
• Based off ezkl's README

`fix_verifier_sol.py` will not be called if ezkl is run from a different directory

For the .sol file to be generated by create-evm-verifier, fix_verifier_sol.py must be present at the directory where ezkl is executed.

To replicate the issue, run ezkl out of the repo, and run the prove and create-evm-verifier function. An empty .sol file will be generated.

"RANGE" gate not satisfied

Describe the bug

When mock-proving with parameters output with calibrate-settings, I get a bunch of failing constraints, e.g.:

error: constraint not satisfied

  Cell layout in region 'model':
    | Offset | A1 | A2 |
    +--------+----+----+
    | 295519 | x0 | x1 | <--{ Gate 'RANGE' applied here

  Constraint '':
    ((S6 * (0x2 - S6)) * (0x3 - S6)) * (x0 - x1) = 0

  Assigned cell values:
    x0 = 0x30644e72e131a029b85045b68181585d2833e84879b9709143e1f593efff790c
    x1 = 0x30644e72e131a029b85045b68181585d2833e84879b9709143e1f593efffa722

Expected behaviors

Not sure why this fails, at least I would need some instructions as to what to change.

Since it does work with other parameters, it also seems like calibrate-settings outputs invalid parameters.

Steps to reproduce the bug

• I'm using lenet.onnx, and input.json, both generated with this Python script
• Run ezkl calibrate-settings -M models/lenet.onnx -D models/input.json --settings-path settings.json
• Run ezkl mock -M models/lenet.onnx -D models/input.json --settings-path settings.json

The generated settings.json is:

{
    "run_args": {
        "tolerance": {
            "Abs": {
                "val": 0
            }
        },
        "scale": 5,
        "bits": 19,
        "logrows": 20,
        "batch_size": 1,
        "input_visibility": "Private",
        "output_visibility": "Public",
        "param_visibility": "Private",
        "pack_base": 1,
        "allocated_constraints": null
    },
    "num_constraints": 295510,
    "model_instance_shapes": [
        [
            1,
            10
        ]
    ],
    "num_hashes": 0,
    "required_lookups": [
        {
            "ReLU": {
                "scale": 32
            }
        }
    ],
    "check_mode": "SAFE"
}

The generated settings seem to contribute to the problem, because the following line works:

ezkl mock -M models/lenet.onnx -D models/input.json --scale 7 --bits 23 --logrows 24

Device and Operating System

OS: Ubuntu
Version: 22.04.2 LTS
Processor: Intel (Hetzner CX51 instance)
Memory: 32 GB

Additional Information

Used the (at the time of writing) most recent EZKL commit: 3dee133

Add config.json invocation option

Currently, we construct the invocation config from either an environment variable or the command line flags (this config json is displayed just after the banner). As the command line gets more complex and tuned to a specific use case, these flags can be annoying to track. A better UX for some purposes would be an ezkl fromconfig command which takes one argument: a config.json file with exactly this configuration json. These config files can be tracked in version control in the same directory as the relevant network.onnx and input.json.

Make it easier to generate an input.json (including the output) from an onnx file and the input

• from an onnx file and the input
• generate a dummy input (and output) given just the onnx file and the input dimensions

Fill in VarTensor and ValTensor combinations

• Some layers are not fully implemented in that they can't handle some combinations of VarTensors and ValTensors
  (see for example convolution) and panic when hitting one of these unsupported combinations.

Ops to implement

• Max
• Max pooling
• Average pooling (a specialization of conv)
• dynamic scaling, and special cases such as softmax

Long verification times with python bindings

Describe the bug

Occasional long verification time with python bindings bindings reported by hadi esna on Telegram.

To Reproduce

1. Build the python bindings for ezkl_lib in python
2. Use the following files Archive.zip

Expected behavior

Verification is not expected to take more than 85mins as reported.

Device

OS: macOS
Version: Ventura 13.3.1 (a)
Processor: Apple M1
Memory: 16 GB

Use dynamic lookups

zcash/halo2#660 is on track to being merged into halo2 soon. We should look into these updates, which enable advice column lookups, into ezkl ASAP.

Partially because they allow for the circumvention of the awkward hybrid lookup / polynomial constraints that have been made in #92 to enable operations such as prelu.

Benefits are:

• simpler implementation of complex nonlinearities like prelu, which have learned weights combined with non-linear functions.
• might enable the removal of the bits cli flag and function arguments (1 less design choice).
• lookups for non-inearities are currently a bottleneck in terms of column size (we need to instantiate table columns with all possible inputs (as dictates by bits)). Dynamic lookup will give us more flexibility in terms of the shape of those columns.

Improvements to library architecture

Now that we have a POC up and running we can work on improving the layout of the library.

• TODO: break down src into sub-packages (addressed by #10)
• TODO: extract trait and structs/types for Tensors eg. Tensor2D or Tensor3D etc... (addressed by #12)
• TODO: extract individual operations, like matmul, sum into functions within tensor_ops; perhaps as function (addressed by #12)
• TODO: Create trait for layers with methods such as forward that are then defined for each layer type
  implmentations on the aforementioned Tensor structs. (addressed by #13)
• TODO: Better parameter loading. (addressed by #14)

" Output mismatch " Error with simple onnx file

Describe the bug

Here is a simple LSTM model attached.
When I try to use it in ezkl it returns this error

[E] [0s, ezkl] - failed: Translating node #70 "_fc1_MatMul" MatMulInference ToTypedTranslator
Error: Translating node #70 "_fc1_MatMul" MatMulInference ToTypedTranslator

Caused by:
    Output mismatch after rewiring expansion for output #0: expected 128,F32 got 1,128,F32

Expected behaviors

Not to return an error

Steps to reproduce the bug

1. build ezkl with -release tag
2. Run ezkl table -M network.onnx

Device and Operating System

OS: MacOS
Version: Ventura 13.3.1
Processor: Apple M1
Memory: 16 GB

Additional Information

network.onnx.zip

Documentation

Currently only Tensor has doc strings. As we start to cement the structure of types we should look to apply the same treatment to other structs and traits.

aggregated proof verification in Solidity

Create a repository of already deployed Verifier.sol so redeployment isn't needed

Is your proposal related to a problem?

Verifier.sol is a view function that smart contracts can reference onchain. If a Verifier.sol has been deployed onchain, another similar one does not need to be deployed. There should be a service that keeps track of this so that end users don't need to redeploy existing Verifiers.

Describe the solution you'd like

Have a hardhat or foundry plugin that checks for Verifier.sol that have been deployed. If the bytecode of the local Verifier.sol matches with something already deployed onchain, inform a user of its existence, address, and chain where it is deployed.

The user should then decide whether to point to that deployed contract in their implementation instead. As Verifier.sol is a view function it can be reused.

A database of such deployments will have to be maintained unfortunately. Users who deploy Verifier.sol onchain could submit their contract into the database.

Make it easy to use srs from public ceremonies

Make it easy to load existing ceremony ptau files for security and convenience.

Automated scaling

We should add automation that chooses reasonable values for scale, logrows, and bits in most cases.

Ubuntu SIGINT on conv2d_mnist example run

Running target/release/examples/conv2d_mnist
The first digit is a 5.0
VK took 204
PK took 125
Killed

trying now with rust version 1.66.0-nightly

a no go even with maxing out cores cargo run --release --example conv2d_mnist --jobs 8

facing "called `Option::unwrap()` on a `None" exception in "src/graph/model.rs"

Describe the bug

I faced this error when trying to run the "setup" step with attached onnx file.

[W] [0s, ezkl_lib::graph::utilities] - matching einsum as a matmul operation
[W] [0s, ezkl_lib::graph::utilities] - matching einsum as a matmul operation
...
[W] [0s, ezkl_lib::graph::utilities] - matching einsum as a matmul operation // around 20 of these
...
[W] [0s, ezkl_lib::graph::utilities] - matching einsum as a matmul operation
[W] [0s, ezkl_lib::graph::utilities] - Unknown op: Concat
[W] [0s, ezkl_lib::graph::utilities] - Unknown op: Slice
[W] [0s, ezkl_lib::graph::utilities] - matching einsum as a matmul operation
[W] [0s, ezkl_lib::graph::utilities] - matching einsum as a matmul operation
[*] [0s, ezkl_lib::graph::model] - calculating num of constraints using dummy model layout...
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/graph/model.rs:897:26
stack backtrace:
   0: rust_begin_unwind
             at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/core/src/panicking.rs:65:14
   2: core::panicking::panic
             at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/core/src/panicking.rs:114:5
   3: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
   4: ezkl_lib::graph::model::Model<F>::dummy_layout_nodes
   5: ezkl_lib::graph::model::Model<F>::gen_params
   6: ezkl_lib::graph::ModelCircuit<F>::new
   7: ezkl_lib::graph::ModelCircuit<F>::from_arg
   8: ezkl_lib::execute::create_keys_kzg
   9: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  10: tokio::runtime::scheduler::current_thread::Context::enter
  11: tokio::macros::scoped_tls::ScopedKey<T>::set
  12: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
  13: tokio::runtime::runtime::Runtime::block_on
  14: ezkl::main

Expected behaviors

Steps to reproduce the bug

1. build the main branch
2. Execute this command with the attached file
   ezkl setup -D input.json -M network.onnx --params-path=../kzg.params --vk-path=vk.key --pk-path=pk.key --circuit-params-path=circuit.params

Device and Operating System

OS: MacOS
Version: Ventura 13.3.1
Processor: Apple M1
Memory: 16 GB

Additional Information

Archive.zip

ff CLI example errors with "lookup input does not exist in table"

I attempted to run

cargo run --release --bin ezkl -- mock -D ./examples/onnx_models/ff_input.json -M ./examples/onnx_models/ff.onnx

in CLI, following instructions in the README. Got this error:

error: lookup input does not exist in table
  (L0, L1) ∉ (F0, F1)

  Lookup 'lk' inputs:
    L0 = x1 * x0 + (1 - x1) * 0
    ^
    | Cell layout in region 'Elementwise':
    |   | Offset | A1 | F2 |
    |   +--------+----+----+
    |   |    0   | x0 | x1 | <--{ Lookup 'lk' inputs queried here
    |
    | Assigned cell values:
    |   x0 = 0x29d8
    |   x1 = 1

    L1 = x1 * x0 + (1 - x1) * 0
    ^
    | Cell layout in region 'Elementwise':
    |   | Offset | A1 | F2 |
    |   +--------+----+----+
    |   |    0   |    | x1 | <--{ Lookup 'lk' inputs queried here
    |   |    1   | x0 |    |
    |
    | Assigned cell values:
    |   x0 = 0x54
    |   x1 = 1

error: lookup input does not exist in table
  (L0, L1) ∉ (F0, F1)

  Lookup 'lk' inputs:
    L0 = x1 * x0 + (1 - x1) * 0
    ^
    | Cell layout in region 'Elementwise':
    |   | Offset | A2 | F2 |
    |   +--------+----+----+
    |   |    0   | x0 | x1 | <--{ Lookup 'lk' inputs queried here
    |
    | Assigned cell values:
    |   x0 = 0x40000000000000000000000000000000224698fc094cf91b992d30ecffffa742
    |   x1 = 1

    L1 = x1 * x0 + (1 - x1) * 0
    ^
    | Cell layout in region 'Elementwise':
    |   | Offset | A2 | F2 |
    |   +--------+----+----+
    |   |    0   |    | x1 | <--{ Lookup 'lk' inputs queried here
    |   |    1   | x0 |    |
    |
    | Assigned cell values:
    |   x0 = 0
    |   x1 = 1

thread 'main' panicked at 'circuit was not satisfied', /Users/bgu/.cargo/git/checkouts/halo2-afe2d0b0be6b3c3a/0cccba0/halo2_proofs/src/dev.rs:1292:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Error while working with Onnx files generated by Pyezkl

I am following the tutorial in Pyezkl repo. I am able to generate the network.onnx and inputs file. However running the following command to generate the proof:
RUST_LOG=debug cargo run --bin ezkl -- --tolerance 2 --scale 4 --bits 16 -K 17 mock -D ./input.json -M ./network.onnx
results in error:
dimensionality error when manipulating a tensor

Here is the complete stack trace for the error:

[*] [0s, ezkl_lib::commands] - loading data from ./input.json
[*] [0s, ezkl_lib::graph::model] - visibility: (inputs: private, params: private, outputs: public)
[*] [0s, ezkl_lib::graph::node] - ------ scaled op node input 0: 7 -> 21
[*] [0s, ezkl_lib::graph::node] - ------ scaled op node input 9: 7 -> 21
[*] [0s, ezkl_lib::graph::node] - ------ scaled const node 4: 7 -> 28
[*] [0s, ezkl_lib::pfsys] - public inputs lengths: [75, 12]
[*] [0s, ezkl_lib::pfsys] - loading params from "kzg.params"
[*] [0s, ezkl_lib::execute] - downsizing params to 17 logrows
[*] [0s, ezkl_lib::graph::model] - visibility: (inputs: private, params: private, outputs: public)
[*] [0s, ezkl_lib::graph::node] - ------ scaled op node input 0: 7 -> 21
[*] [0s, ezkl_lib::graph::node] - ------ scaled op node input 9: 7 -> 21
[*] [0s, ezkl_lib::graph::node] - ------ scaled const node 5: 7 -> 28
[*] [0s, ezkl_lib::graph] - total var len: 8470
[*] [0s, ezkl_lib::graph] - number of advices used: 3
[*] [0s, ezkl_lib::graph] - number of fixed used: 0
[*] [0s, ezkl_lib::graph] - number of instances used: 2
[*] [0s, ezkl_lib::graph::model] - configuring model
[W] [0s, ezkl_lib::circuit] - assiging lookup input
[*] [0s, ezkl_lib::graph::model] - model layout
[W] [0s, ezkl_lib::tensor::val] - using 'get_inner' in constraints can create soundness issues.
[E] [0s, ezkl_lib::graph::model] - dimensionality error when manipulating a tensor
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Synthesis', Desktop/git/ezkl/src/graph/mod.rs:131:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Make it easy to use RawBytes parameters

Previously we supported the "native" RawBytes params, which are in a format incompatible with public ceremonies, but are faster to load and save. Now we support loading public ceremony params, but have to pay the deserialization overhead which makes loading take longer. We could cache the RawBytes-format params after reading from a public setup, and reuse them on subsequent calls, or suggest running a conversion script. Ideally a solution would allow for the faster format, but not require additional steps.

Accumulation arguments

We currently support two types of arguments (each used for many nodes), polynomial and lookup. Significant fusion is possible within each argument type.

To provide an alternative performance tradeoff, general accumulation arguments (such as for dot product using an accumulator m and the gate $m_i = a_i*b_i + m_{i-1}$) should also be supported. For some operations, we expect that accumulation will be significantly faster (e.g. it can use few columns and few rotations), and in any case it provides a different cost profile than either type of argument we currently use, which should enable ezkl to work for more use cases.

Once we get these working with the ezkl node api, there are some further tricks that should be available to make more progress on performance.

`prove` and `verify` for evm pipeline

Currently the evm pipeline (activated with --features evm ) is only enabled for fullprove whereby the proof and verifying key (and other parameters) are keep in memory and never serialized to disk !

We need to similarly enable separate prove and verify commands specific to evm verification ! When calling prove this would involve serializing the proof generated by

let proof = gen_kzg_proof::<
                        _,
                        _,
                        EvmTranscript<G1Affine, _, _, _>,
                        EvmTranscript<G1Affine, _, _, _>,
                    >(
                        &params, &pk, agg_circuit.clone(), agg_circuit.instances()
                    )?;

the aggregate circuit instances generated from:

let instances = agg_circuit.instances();

and the verification smart contract deployment code:

let deployment_code = gen_aggregation_evm_verifier(
                        &params,
                        pk.get_vk(),
                        AggregationCircuit::num_instance(),
                        AggregationCircuit::accumulator_indices(),
                    )?;

Calling verifywould then read these serialized values from disk and call:

evm_verify(deployment_code, agg_circuit.instances(), proof)?;

In summary this involves the following tasks:

• separate prove and verify pathways in src/execute.rs for when --features evm is activated (for a template for how this is done, see fullprove in the same file)
• serializing the aggregate circuit instances and saving to a file
• serializing the aggregate circuit proof and saving to a file
• serializing the verification smart contract deployment code and saving to file
• verify should read all 3 of the above files and call evm_verify
• add integration tests which test the new pipeline

simple evm pipeline

• Current EVM verification pipeline involves the use of accumulation / proof aggregation. Although this allows us to verify pretty large circuits on the EVM, for smaller circuits the proof generation time vs proof size benefits may be lesser.
• We need an additional pipeline with which we can straighforwardly verify (small) proofs using the EVM without using accumulation.

To do so we should

• create a new evm submodule within ./src/pfsys and move the existing aggregation.rs file into it.
• create a new file which does not use proof aggregation, we can perhaps take inspiration from snark-verifier
• move functions that are common to the aggregation and simple pipelines into ./src/pfsys/evm/mod.rs
• add a cli flag which triggers the usage (or not) of proof aggregation when using the evm verifier
• add integration tests to test the new pipeline

Outputs of EZKL goes into stderr pipe instead of stdout pipe

I came across this minor issue while working on automation with ezkl with python's subprocess. Seems like the outputs of EZKL goes into stderr instead of stdout.

Outputs from the python automation script

{
	"stderr": "[*] \n |          ███████╗███████╗██╗  ██╗██╗\n |          ██╔════╝╚══███╔╝██║ ██╔╝██║\n |          █████╗    ███╔╝ █████╔╝ ██║\n |          ██╔══╝   ███╔╝  ██╔═██╗ ██║\n |          ███████╗███████╗██║  ██╗███████╗\n |          ╚══════╝╚══════╝╚═╝  ╚═╝╚══════╝\n |  \n |          -----------------------------------------------------------\n |          Easy Zero Knowledge for Layers.\n |          -----------------------------------------------------------\n |          \n[*] {\"command\":{\"Prove\":{\"data\":\"/app/inputdata/92ccc9d7-a297-41a4-903c-cf4040ca7126.json\",\"model\":\"/app/onnxmodel/6bf2fb85-7b9a-4cf3-8670-5cfed8b53053.onnx\",\"vk_path\":\"/app/generated/inputdata_92ccc9d7-a297-41a4-903c-cf4040ca7126+onnxmodel_6bf2fb85-7b9a-4cf3-8670-5cfed8b53053.vk\",\"proof_path\":\"/app/generated/inputdata_92ccc9d7-a297-41a4-903c-cf4040ca7126+onnxmodel_6bf2fb85-7b9a-4cf3-8670-5cfed8b53053.pf\",\"params_path\":\"/app/kzg.params\",\"transcript\":\"EVM\",\"strategy\":\"Single\"}},\"args\":{\"tolerance\":0,\"scale\":7,\"bits\":16,\"logrows\":17,\"public_inputs\":false,\"public_outputs\":true,\"public_params\":false,\"max_rotations\":512}}\n[*] loading data from /app/inputdata/92ccc9d7-a297-41a4-903c-cf4040ca7126.json\n[*] visibility: (inputs: private, params: private, outputs: public)\n[*] ------ scaled const node 1: 7 -> 14\n[*] assigning configuration buckets to operations\n[*] public inputs lengths: [1]\n[*] loading params from \"/app/kzg.params\"\n[*] visibility: (inputs: private, params: private, outputs: public)\n[*] ------ scaled const node 2: 7 -> 14\n[*] assigning configuration buckets to operations\n[*] number of advices used: 4\n[*] number of fixed used: 0\n[*] number of instances used: 1\n[*] configuring model\n[*] output_shapes [[1]]\n[*] model layout\n[*] model outputs are nodes: [4]\n[*] computing...\n[*] VK took 1\n[*] visibility: (inputs: private, params: private, outputs: public)\n[*] ------ scaled const node 2: 7 -> 14\n[*] assigning configuration buckets to operations\n[*] number of advices used: 4\n[*] number of fixed used: 0\n[*] number of instances used: 1\n[*] configuring model\n[*] output_shapes [[1]]\n[*] model layout\n[*] model outputs are nodes: [4]\n[*] computing...\n[*] PK took 1\n[*] visibility: (inputs: private, params: private, outputs: public)\n[*] ------ scaled const node 2: 7 -> 14\n[*] assigning configuration buckets to operations\n[*] number of advices used: 4\n[*] number of fixed used: 0\n[*] number of instances used: 1\n[*] configuring model\n[*] output_shapes [[1]]\n[*] model layout\n[*] model outputs are nodes: [4]\n[*] computing...\n[*] visibility: (inputs: private, params: private, outputs: public)\n[*] ------ scaled const node 2: 7 -> 14\n[*] assigning configuration buckets to operations\n[*] number of advices used: 4\n[*] number of fixed used: 0\n[*] number of instances used: 1\n[*] configuring model\n[*] output_shapes [[1]]\n[*] model layout\n[*] model outputs are nodes: [4]\n[*] computing...\n[*] Proof took 5\n[*] verify took 0\n[*] proof took 5\n[*] saving verification key 💾\n[*] succeeded\n",
	"stdout": ""
}