jackoelv/bellperson

zk-SNARK library

Contributors

• [FOO]jackoelv
• [FOO]ROy
• [FOO]Gabe
• [FOO-Hacker]Jerry
• [FOO-TEST]我是鱼饵
• [FOO-C2]devid
• [FOO]石头
• [FOO]Waitting for you ...

Add 3080 Branch

• to do...

Add 3090 Support

• 500s+ when after all optimization, should be 400s+
• the branch 3090 works now.
• need team work to finish the code to save more time.

to do list:

1. prover.rs: 425 change to run multi fft on one 3090 card.
2. provers.rs: 500, density_filter change to parallel.
3. provers.rs:305, synthesize function change to multi cpu parallel.
4. provers.rs:425 change to run multi gpu( later I will upload a multi gpu branch, change a little can work.)
5. gpu params opimization: src/gpu/multiexp.rs 52:calc_num_groups 76:calc_best_chunk_size 87:calc_chunk_size
6. change openCL to CUDA.

C2 in 880s tested.

• this is special for 2080Ti, may not work with any other GPU cards. because different gpu have different cuda-cores and mem.
• some calculations with cpus change to parallel.
• change params special for 2080Ti. because it has 68 SMs and 64 SP per SM.
• we are going on with more optimizations. May 100s even more can be saved.
• Team work requests are welcome!

From ZQBC

HOWTO

• get the code:

cd ../lotus_code_path && git clone https://github.com/jackoelv/bellperson.git && git checkout origin/2080Ti

• patch the filecoin-ffi submodule

cd ./lotus_code_path && git submodule update --init --recursive
cd ./lotus_code_path/extern/filecoin-ffi/rust/
vi Cargo.toml 

in the end of the file add patch code:

[patch.crates-io]
bellperson = { path = "../../../../bellperson" }

• then update cargo package

cd ./lotus_code_path/extern/filecoin-ffi/rust/
cargo update
cd ./lotus_code_path
RUSTFLAGS="-C target-cpu=native -g" FFI_BUILD_FROM_SOURCE=1 make clean all

enjoy it!

DONATION

Jennifer suggested that I should have a donation wallet.

fils are welcome if you like.

my wallet addr:

f1ki5mgbm4cyz43oamnbvv5bjrqdsvkphuxxs2h4a

FAQ

openCL error

if you see error like this:

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

because the MEM needs exceeds maxinum gpu mem. I am looking for new solution to caculate the suitable mem needs. a temporary solution is change the variable ：

• src/gpu/multiexp.rs

324: jack_chunk = (jack_chunk as f64 / 10f64).ceil() as usize;

you can increase 10f64 to 11f64 even bigger to reduce mem needs. I am trying a new solution which change window_size for lower cost of mem. In prover.rs, b_g2_aux caculation size of <::Projective> is doubled, so with same window_size and num_groups, mem needs increase fast.

attention: this code removed from 3090 branch.

bellperson

This is a fork of the great bellman library.

bellman is a crate for building zk-SNARK circuits. It provides circuit traits and primitive structures, as well as basic gadget implementations such as booleans and number abstractions.

Backend

There are currently two backends available for the implementation of Bls12 381:

• paired - pure Rust implementation
• blstrs - optimized with hand tuned assembly, using blst

They can be selected at compile time with the mutually exclusive features pairing and blst. Specifying one of them is enough for a working library, no additional features need to be set. The default for now is pairing, as the secure and audited choice.

GPU

This fork contains GPU parallel acceleration to the FFT and Multiexponentation algorithms in the groth16 prover codebase under the compilation feature gpu, it can be used in combination with pairing or blst.

Requirements

• NVIDIA or AMD GPU Graphics Driver
• OpenCL

( For AMD devices we recommend ROCm )

Environment variables

The gpu extension contains some env vars that may be set externally to this library.

• BELLMAN_NO_GPU

  Will disable the GPU feature from the library and force usage of the CPU.

  // Example
  env::set_var("BELLMAN_NO_GPU", "1");

• BELLMAN_VERIFIER

  Chooses the device in which the batched verifier is going to run. Can be cpu, gpu or auto.

  Example
  env::set_var("BELLMAN_VERIFIER", "gpu");

• BELLMAN_CUSTOM_GPU

  Will allow for adding a GPU not in the tested list. This requires researching the name of the GPU device and the number of cores in the format ["name:cores"].

  // Example
  env::set_var("BELLMAN_CUSTOM_GPU", "GeForce RTX 2080 Ti:4352, GeForce GTX 1060:1280");

• BELLMAN_CPU_UTILIZATION

  Can be set in the interval [0,1] to designate a proportion of the multiexponenation calculation to be moved to cpu in parallel to the GPU to keep all hardware occupied.

  // Example
  env::set_var("BELLMAN_CPU_UTILIZATION", "0.5");

Supported / Tested Cards

Depending on the size of the proof being passed to the gpu for work, certain cards will not be able to allocate enough memory to either the FFT or Multiexp kernel. Below are a list of devices that work for small sets. In the future we will add the cuttoff point at which a given card will not be able to allocate enough memory to utilize the GPU.

Device Name	Cores	Comments
Quadro RTX 6000	4608
TITAN RTX	4608
Tesla V100	5120
Tesla P100	3584
Tesla T4	2560
Quadro M5000	2048
GeForce RTX 2080 Ti	4352
GeForce RTX 2080 SUPER	3072
GeForce RTX 2080	2944
GeForce RTX 2070 SUPER	2560
GeForce GTX 1080 Ti	3584
GeForce GTX 1080	2560
GeForce GTX 2060	1920
GeForce GTX 1660 Ti	1536
GeForce GTX 1060	1280
GeForce GTX 1650 SUPER	1280
GeForce GTX 1650	896
gfx1010	2560	AMD RX 5700 XT

Running Tests

To run the multiexp_consistency test you can use:

RUST_LOG=info cargo test --features gpu -- --exact multiexp::gpu_multiexp_consistency --nocapture

Considerations

Bellperson uses rust-gpu-tools as its OpenCL backend, therefore you may see a directory named ~/.rust-gpu-tools in your home folder, which contains the compiled binaries of OpenCL kernels used in this repository.

License

Licensed under either of

• Apache License, Version 2.0, |LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

bellperson's Issues

3090 问题

最新版本有4个branch，好像没有z-0.13.0-3090版本
所以rust$ cargo update 时候报如下

    Updating `git://mirrors.ustc.edu.cn/crates.io-index` index
warning: Patch `bellperson v0.12.3 (/home/lotus/source/bellperson.3080)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.

是不是这个版本没法支持3090 呢？

CPU7542显卡3090下编译运行时的错误

在docker下编译，编译没有问题，运行lotus-bench sealing --storage-dir=/home/lotus_user --sector-size 512Mib出错

2021-01-27T14:04:03.948 TRACE storage_proofs_porep::stacked::vanilla::proof > node index 800000/800000/16777216
2021-01-27T14:04:13.773 DEBUG rust_gpu_tools::opencl::utils > loaded devices: {}
thread '' panicked at 'failed to create ColumnTreeBuilder: ClError(BusIdNotAvailable)', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/storage-proofs-porep-5.4.0/src/stacked/vanilla/proof.rs:516:23
stack backtrace:
0: 0x1e95180 - std::backtrace_rs::backtrace::libunwind::trace::he85dfb3ae4206056
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/../../backtrace/src/backtrace/libunwind.rs:96
1: 0x1e95180 - std::backtrace_rs::backtrace::trace_unsynchronized::h1ad28094d7b00c21
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/../../backtrace/src/backtrace/mod.rs:66
2: 0x1e95180 - std::sys_common::backtrace::_print_fmt::h901b54610713cd21
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:79
3: 0x1e95180 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hb0ad78ee1571f7e0
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:58
4: 0x1f0393c - core::fmt::write::h1857a60b204f1b6a
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/fmt/mod.rs:1080
5: 0x1e86e32 - std::io::Write::write_fmt::hf7b7d7b243f84a36
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/io/mod.rs:1516
6: 0x1e99f5d - std::sys_common::backtrace::_print::hd093978a5287b8ff
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:61
7: 0x1e99f5d - std::sys_common::backtrace::print::h20f46787581d56d7
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:48
8: 0x1e99f5d - std::panicking::default_hook::{{closure}}::h486cbb4b82ffc357
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:208
9: 0x1e99c08 - std::panicking::default_hook::h4190c9e3edd4d591
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:227
10: 0x1e9a781 - std::panicking::rust_panic_with_hook::h72e78719cdda225c
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:577
11: 0x1e9a329 - std::panicking::begin_panic_handler::{{closure}}::h8bd07dbd34150a96
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:484
12: 0x1e9560c - std::sys_common::backtrace::__rust_end_short_backtrace::hdb6b3066ad29028a
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:153
13: 0x1e9a2e9 - rust_begin_unwind
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:483
14: 0x1effc61 - core::panicking::panic_fmt::hb15d6f55e8472f62
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/panicking.rs:85
15: 0x1eff8d3 - core::option::expect_none_failed::he492a18657d97593
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/option.rs:1234
16: 0x161d025 - <std::panic::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once::h05f8c133988411b0
17: 0x1f3d2de - std::panicking::try::h738a6aefcb1c5213
18: 0x179e289 - <rayon_core::job::HeapJob as rayon_core::job::Job>::execute::h8cb396f288390cdb
19: 0x1e22b51 - rayon_core::registry::WorkerThread::wait_until_cold::hd677f2fc93300658
20: 0x1e1e5f4 - rayon_core::scope::ScopeBase::steal_till_jobs_complete::h936fa42c02e8ad82
21: 0x163b4fd - <std::panic::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once::h86dce894170249fd
22: 0x17ab536 - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::h7ed5219388004138
23: 0x1e22b51 - rayon_core::registry::WorkerThread::wait_until_cold::hd677f2fc93300658
24: 0x1e20d92 - rayon_core::registry::ThreadBuilder::run::h055ee17e713b5ed1
25: 0x1e26dc3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h39369de0c1c71746
26: 0x1e282f4 - core::ops::function::FnOnce::call_once{{vtable.shim}}::ha9f50c0602ee0218
27: 0x1ea90ba - <alloc::boxed::Box as core::ops::function::FnOnce>::call_once::h1080dfe0ef616bdf
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/alloc/src/boxed.rs:1042
28: 0x1ea90ba - <alloc::boxed::Box as core::ops::function::FnOnce>::call_once::hd2747e1f2d5cec32
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/alloc/src/boxed.rs:1042
29: 0x1ea90ba - std::sys::unix::thread::Thread::new::thread_start::hd0f336b4ef6808a7
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys/unix/thread.rs:87
30: 0x7f741d3f76db - start_thread
31: 0x7f741dcd671f - __clone
32: 0x0 -
thread '' panicked at 'failed to send columns: "SendError(..)"', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/storage-proofs-porep-5.4.0/src/stacked/vanilla/proof.rs:502:34
stack backtrace:
0: 0x1e95180 - std::backtrace_rs::backtrace::libunwind::trace::he85dfb3ae4206056
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/../../backtrace/src/backtrace/libunwind.rs:96
1: 0x1e95180 - std::backtrace_rs::backtrace::trace_unsynchronized::h1ad28094d7b00c21
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/../../backtrace/src/backtrace/mod.rs:66
2: 0x1e95180 - std::sys_common::backtrace::_print_fmt::h901b54610713cd21
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:79
3: 0x1e95180 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hb0ad78ee1571f7e0
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:58
4: 0x1f0393c - core::fmt::write::h1857a60b204f1b6a
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/fmt/mod.rs:1080
5: 0x1e86e32 - std::io::Write::write_fmt::hf7b7d7b243f84a36
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/io/mod.rs:1516
6: 0x1e99f5d - std::sys_common::backtrace::_print::hd093978a5287b8ff
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:61
7: 0x1e99f5d - std::sys_common::backtrace::print::h20f46787581d56d7
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:48
8: 0x1e99f5d - std::panicking::default_hook::{{closure}}::h486cbb4b82ffc357
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:208
9: 0x1e99c08 - std::panicking::default_hook::h4190c9e3edd4d591
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:227
10: 0x1e9a781 - std::panicking::rust_panic_with_hook::h72e78719cdda225c
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:577
11: 0x1e9a329 - std::panicking::begin_panic_handler::{{closure}}::h8bd07dbd34150a96
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:484
12: 0x1e9560c - std::sys_common::backtrace::__rust_end_short_backtrace::hdb6b3066ad29028a
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys_common/backtrace.rs:153
13: 0x1e9a2e9 - rust_begin_unwind
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:483
14: 0x1effc61 - core::panicking::panic_fmt::hb15d6f55e8472f62
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/panicking.rs:85
15: 0x1eff8d3 - core::option::expect_none_failed::he492a18657d97593
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/option.rs:1234
16: 0x16252ac - <std::panic::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once::h27ffb426a9990e61
17: 0x1f3cd4e - std::panicking::try::h6dee0badf58d0b8d
18: 0x179d5b0 - <rayon_core::job::HeapJob as rayon_core::job::Job>::execute::h5aef4a5ebaf0e015
19: 0x1e22b51 - rayon_core::registry::WorkerThread::wait_until_cold::hd677f2fc93300658
20: 0x1e20d92 - rayon_core::registry::ThreadBuilder::run::h055ee17e713b5ed1
21: 0x1e26dc3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h39369de0c1c71746
22: 0x1e282f4 - core::ops::function::FnOnce::call_once{{vtable.shim}}::ha9f50c0602ee0218
23: 0x1ea90ba - <alloc::boxed::Box as core::ops::function::FnOnce>::call_once::h1080dfe0ef616bdf
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/alloc/src/boxed.rs:1042
24: 0x1ea90ba - <alloc::boxed::Box as core::ops::function::FnOnce>::call_once::hd2747e1f2d5cec32
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/alloc/src/boxed.rs:1042
25: 0x1ea90ba - std::sys::unix::thread::Thread::new::thread_start::hd0f336b4ef6808a7
at /rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/sys/unix/thread.rs:87
26: 0x7f741d3f76db - start_thread
27: 0x7f741dcd671f - __clone
28: 0x0 -
2021-01-27T14:04:14.054+0800 WARN lotus-bench lotus-bench/main.go:116 failed to run seals:
main.glob..func3
/go/lotus/cmd/lotus-bench/main.go:277

• commit:
  main.runSeals.func1.1
  /go/lotus/cmd/lotus-bench/main.go:579
• presealing sector 0 ():
  github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper.(*Sealer).SealPreCommit2
  /go/lotus/extern/sector-storage/ffiwrapper/sealer_cgo.go:530
• Rust panic: no unwind information
  github.com/filecoin-project/filecoin-ffi.SealPreCommitPhase2
  /go/lotus/extern/filecoin-ffi/proofs.go:321
  github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper.(*Sealer).SealPreCommit2
  /go/lotus/extern/sector-storage/ffiwrapper/sealer_cgo.go:528
  main.runSeals.func1.1
  /go/lotus/cmd/lotus-bench/main.go:577
  main.runSeals.func1
  /go/lotus/cmd/lotus-bench/main.go:688
  runtime.goexit
  /usr/local/go/src/runtime/asm_amd64.s:1374

编译中的一些warning

warning: unused variable: exp_bits
--> /home/bugx/lotus/bellperson/src/gpu/multiexp.rs:110:13
|
110 | let exp_bits = exp_size::() * 8;
| ^^^^^^^^ help: if this is intentional, prefix it with an underscore: _exp_bits
|
= note: #[warn(unused_variables)] on by default

Compiling reqwest v0.10.10
warning: constant is never used: MAX_WINDOW_SIZE
--> /home/bugx/lotus/bellperson/src/gpu/multiexp.rs:19:1
|
19 | const MAX_WINDOW_SIZE: usize = 12;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(dead_code)] on by default

warning: constant is never used: MEMORY_PADDING
--> /home/bugx/lotus/bellperson/src/gpu/multiexp.rs:21:1
|
21 | const MEMORY_PADDING: f64 = 0.1f64; // Let 20% of GPU memory be free
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: function is never used: calc_best_chunk_size
--> /home/bugx/lotus/bellperson/src/gpu/multiexp.rs:76:4
|
76 | fn calc_best_chunk_size(max_window_size: usize, core_count: usize, exp_bits: usize) -> usize {
| ^^^^^^^^^^^^^^^^^^^^

warning: function is never used: calc_chunk_size
--> /home/bugx/lotus/bellperson/src/gpu/multiexp.rs:87:4
|
87 | fn calc_chunk_size(mem: u64, core_count: usize) -> usize
| ^^^^^^^^^^^^^^^

Compiling phase21 v0.11.0
warning: 5 warnings emitted

z-0.13.0-3080 error[E0432]: unresolved import `heim::memory`

Compiling config v0.10.1
Compiling bls-signatures v0.10.0
Compiling bellperson v0.14.2
Compiling heim-runtime v0.0.7 (/root/lotus/heim/heim-runtime)
Compiling neptune v3.0.0
Compiling fr32 v1.0.2
Compiling heim v0.0.11 (/root/lotus/heim/heim)
Compiling bellperson v0.14.2 (/root/lotus/bellperson)
error[E0432]: unresolved import heim::memory
--> /root/lotus/bellperson/src/groth16/prover.rs:276:12
|
276 | use heim::{memory, units::information};
| ^^^^^^ no memory in the root

Compiling filecoin-hashers v3.0.2
Compiling filcrypto v0.7.5 (/root/lotus/extern/filecoin-ffi/rust)
Compiling storage-proofs-core v8.0.2
error: aborting due to previous error

For more information about this error, try rustc --explain E0432.
error: could not compile bellperson

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

• rm -f /mnt/md0/tmp/tmp.5gLPbYldaK
  Makefile:11: recipe for target '.install-filcrypto' failed
  make[1]: *** [.install-filcrypto] Error 101
  make[1]: Leaving directory '/root/lotus/extern/filecoin-ffi'
  Makefile:37: recipe for target 'build/.filecoin-install' failed
  make: *** [build/.filecoin-install] Error 2
  root@SYS-1-WORKER3-223:~/lotus#

I download the heim package and change it to v0.0.11. I google for a long time and there is no answer to this issue.

lotus version 1.11.1-dev+mainnet+git.2d40089b7

root@SYS-1-WORKER3-223:/lotus# cd bellperson/
root@SYS-1-WORKER3-223:/lotus/bellperson# BELLMAN_NUM_CPUS=1 RUSTFLAGS="-C target-cpu=native" RUST_LOG=debug cargo test --release --all --workspace -- --nocapture
Compiling memchr v2.4.0
Compiling heim-common v0.0.11 (/root/lotus/heim/heim-common)
Compiling futures-util v0.3.15
Compiling aho-corasick v0.7.18
Compiling bstr v0.2.16
Compiling csv-core v0.1.10
Compiling csv v1.1.6
Compiling regex v1.5.4
Compiling criterion v0.3.4
Compiling env_logger v0.8.4
Compiling futures-executor v0.3.15
Compiling futures v0.3.15
Compiling heim-runtime v0.0.7 (/root/lotus/heim/heim-runtime)
Compiling heim v0.0.11 (/root/lotus/heim/heim)
Compiling bellperson v0.14.2 (/root/lotus/bellperson)
error[E0432]: unresolved import heim::memory
--> src/groth16/prover.rs:276:12
|
276 | use heim::{memory, units::information};
| ^^^^^^ no memory in the root

error: aborting due to previous error

For more information about this error, try rustc --explain E0432.
error: could not compile bellperson.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
root@SYS-1-WORKER3-223:~/lotus/bellperson#

I guess the problem is because the variable 'memory' is not defined.

lotus-bench:failed to create ColumnTreeBuilder: ClError(BusIdNotAvailable)

2021-01-26T20:16:44.626 INFO storage_proofs_porep::stacked::vanilla::cache > parent cache: opening /home/lotus_user/filecoin_proofs_parent_cache/v28-sdr-parent-7ba215a1d2345774ab90b8cb1158d296e409d6068819d7b8c7baf0b25d63dc34.cache, verify enabled: false
2021-01-26T20:16:44.626 INFO storage_proofs_porep::stacked::vanilla::proof > multi core replication
2021-01-26T20:16:44.626 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > create labels
2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > Cores: 32, Shared Caches: 8, cores per cache (group_size): 4
2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > checked out core group 0
2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in main thread
2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 0,32
2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 0
2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::memory_handling > initializing cache
2021-01-26T20:16:44.692 WARN storage_proofs_porep::stacked::vanilla::memory_handling > failed to lock map Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }, falling back
2021-01-26T20:16:44.692 WARN storage_proofs_porep::stacked::vanilla::memory_handling > failed to lock map Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }, falling back
2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Layer 1
2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Creating labels for layer 1
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 2
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 3,35
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 3
2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 0
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 1,33
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 1
2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner
2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 1
2021-01-26T20:16:44.700 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 2,34
2021-01-26T20:16:44.700 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 2
2021-01-26T20:16:44.700 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner
2021-01-26T20:16:45.944 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1399425
2021-01-26T20:16:49.873 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 5777665
2021-01-26T20:16:51.161 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 7204993
2021-01-26T20:16:55.505 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 12001921
2021-01-26T20:16:59.855 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > storing labels on disk
2021-01-26T20:17:00.001 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > generated layer 1 store with id layer-1
2021-01-26T20:17:00.001 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Layer 2
2021-01-26T20:17:00.001 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Creating labels for layer 2
2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1
2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 2
2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 3
2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 3
2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 1
2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 2
2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 2
2021-01-26T20:17:00.002 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner
2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 0
2021-01-26T20:17:00.002 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner
2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 1
2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 1
2021-01-26T20:17:00.002 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner
2021-01-26T20:17:00.625 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 702465
2021-01-26T20:17:00.627 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 702849
2021-01-26T20:17:01.638 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1843073
2021-01-26T20:17:05.746 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 6434817
2021-01-26T20:17:11.633 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 12956289
2021-01-26T20:17:15.112 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > storing labels on disk
2021-01-26T20:17:15.256 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > generated layer 2 store with id layer-2
2021-01-26T20:17:15.333 INFO filecoin_proofs::api::seal > seal_pre_commit_phase1:finish: SectorId(0)
2021-01-26T20:17:15.333 INFO filcrypto::proofs::api > seal_pre_commit_phase1: finish
2021-01-26T20:17:15.333+0800 INFO lotus-bench lotus-bench/main.go:576 [0] Running replication(2)...
2021-01-26T20:17:15.333 INFO filcrypto::proofs::api > seal_pre_commit_phase2: start
2021-01-26T20:17:15.333 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:start
2021-01-26T20:17:15.333 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:finish
2021-01-26T20:17:15.333 INFO filecoin_proofs::api::seal > seal_pre_commit_phase2:start
2021-01-26T20:17:15.333 TRACE filecoin_proofs::api::seal > seal phase 2: base tree size 33554431, base tree leafs 16777216, rows to discard 7
2021-01-26T20:17:15.334 INFO storage_proofs_porep::stacked::vanilla::proof > replicate_phase2
2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > transform_and_replicate_layers
2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > nodes count 16777216, data len 536870912
2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > is_merkle_tree_size_valid(16777216, BINARY_ARITY) = true
2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > is_merkle_tree_size_valid(16777216, 8) = true
2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > tree_r_last using rows_to_discard=2
2021-01-26T20:17:15.334 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree c using the GPU
2021-01-26T20:17:15.334 INFO storage_proofs_porep::stacked::vanilla::proof > Building column hashes
2021-01-26T20:17:15.334 INFO neptune::triton::cl > getting default futhark context
2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > processing config 1/1 with column nodes 800000
2021-01-26T20:17:15.334 INFO neptune::triton::cl > getting context for ~Index(0)
2021-01-26T20:17:15.512 TRACE storage_proofs_porep::stacked::vanilla::proof > node index 800000/800000/16777216
2021-01-26T20:17:25.336 DEBUG rust_gpu_tools::opencl::utils > loaded devices: {}
thread '' panicked at 'failed to create ColumnTreeBuilder: ClError(BusIdNotAvailable)', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/storage-proofs-porep-5.4.0/src/stacked/vanilla/proof.rs:516:23
stack backtrace:
0: rust_begin_unwind
at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:483
1: core::panicking::panic_fmt
at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/panicking.rs:85
2: core::option::expect_none_failed
at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/option.rs:1234
3: <std::panic::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once
4: std::panicking::try
5: <rayon_core::job::HeapJob as rayon_core::job::Job>::execute
6: rayon_core::registry::WorkerThread::wait_until_cold
7: rayon_core::scope::ScopeBase::steal_till_jobs_complete
8: rayon_core::scope::scope::{{closure}}
9: std::panicking::try
10: <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute
11: rayon_core::registry::WorkerThread::wait_until_cold
12: rayon_core::registry::ThreadBuilder::run
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.
thread '' panicked at 'failed to send columns: "SendError(..)"', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/storage-proofs-porep-5.4.0/src/stacked/vanilla/proof.rs:502:34
stack backtrace:
0: rust_begin_unwind
at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:483
1: core::panicking::panic_fmt
at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/panicking.rs:85
2: core::option::expect_none_failed
at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/option.rs:1234
3: <std::panic::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once
4: std::panicking::try
5: <rayon_core::job::HeapJob as rayon_core::job::Job>::execute
6: rayon_core::registry::WorkerThread::wait_until_cold
7: rayon_core::registry::ThreadBuilder::run
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.
2021-01-26T20:17:25.659+0800 WARN lotus-bench lotus-bench/main.go:116 failed to run seals:
main.glob..func3
/home/bugx/lotus/cmd/lotus-bench/main.go:277

• commit:
  main.runSeals.func1.1
  /home/bugx/lotus/cmd/lotus-bench/main.go:579
• presealing sector 0 ():
  github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper.(*Sealer).SealPreCommit2
  /home/bugx/lotus/extern/sector-storage/ffiwrapper/sealer_cgo.go:486
• Rust panic: no unwind information
  github.com/filecoin-project/filecoin-ffi.SealPreCommitPhase2
  /home/bugx/lotus/extern/filecoin-ffi/proofs.go:321
  github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper.(*Sealer).SealPreCommit2
  /home/bugx/lotus/extern/sector-storage/ffiwrapper/sealer_cgo.go:484
  main.runSeals.func1.1
  /home/bugx/lotus/cmd/lotus-bench/main.go:577
  main.runSeals.func1
  /home/bugx/lotus/cmd/lotus-bench/main.go:688
  runtime.goexit
  /usr/local/go/src/runtime/asm_amd64.s:1374

Cargo update 时报 Patch `bellperson v0.12.3 (/lotus/bellperson)` was not used in the crate graph.

执行cargo update时报如下警告：

    Updating crates.io index
warning: Patch `bellperson v0.12.3 (/lotus/bellperson)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.

是不是安装失败？

【第一次测试结果】AMD EPYC 7662 64-Core Processor + Nvidia 3090

results (v28) SectorSize:(34359738368), SectorNumber:(1)
seal: addPiece: 1m40.626541161s (325.6 MiB/s)
seal: preCommit phase 1: 3h3m16.757401329s (2.98 MiB/s)
seal: preCommit phase 2: 23m43.763900117s (23.02 MiB/s)
seal: commit phase 1: 332.897593ms (96.13 GiB/s)
seal: commit phase 2: 16m5.706406147s (33.93 MiB/s)
seal: verify: 9.431625ms
unseal: 2h59m19.917478493s (3.045 MiB/s)

generate candidates: 242.142µs (129.1 TiB/s)
compute winning post proof (cold): 6.153315373s
compute winning post proof (hot): 5.995593782s
verify winning post proof (cold): 58.437475ms
verify winning post proof (hot): 4.188863ms

compute window post proof (cold): 3m34.186351049s
compute window post proof (hot): 4m22.816593821s
verify window post proof (cold): 7.206689974s
verify window post proof (hot): 12.468818ms

2080TI Status error code: CL_NV_INVALID_MEM_ACCESS (-9999)

3090 cargo update error

qwe@qwe-server:~/test/lotus/extern/filecoin-ffi/rust$ cargo update Updating https://mirrors.tuna.tsinghua.edu.cn/git/crates.io-index.git` index
error: failed to select a version for the requirement zeroize = "^0.9.3"
candidate versions found which didn't match: 1.3.0, 1.2.0, 1.1.1, ...
location searched: https://mirrors.tuna.tsinghua.edu.cn/git/crates.io-index.git index (which is replacing registry https://github.com/rust-lang/crates.io-index)
required by package dialoguer v0.7.1
... which is depended on by filecoin-proofs v5.4.0
... which is depended on by filecoin-proofs-api v5.4.1
... which is depended on by filcrypto v0.7.5 (/home/qwe/test/lotus/extern/filecoin-ffi/rust)
perhaps a crate was updated and forgotten to be re-vendored?`

Lotus is 1.6.0.
How should I fix this problem?

【amd7542+3080测试结果】

results (v28) SectorSize:(34359738368), SectorNumber:(1)
seal: addPiece: 2m14.487987532s (243.7 MiB/s)
seal: preCommit phase 1: 3h4m11.019860994s (2.965 MiB/s)
seal: preCommit phase 2: 37m8.059063771s (14.71 MiB/s)
seal: commit phase 1: 524.134053ms (61.05 GiB/s)
seal: commit phase 2: 18m7.608299245s (30.13 MiB/s)
seal: verify: 15.681922ms
unseal: 2h57m44.842597962s (3.073 MiB/s)

是否支持lotus 1.11.1版本？

显卡2080Ti，请问现在最高支持到lotus什么版本呢？

lotus-bench 裸机下运行错误

Ubuntu 20.04
3090显卡
cpu 7542

2021-01-27T16:35:11.829+0800 INFO lotus-bench lotus-bench/main.go:606 [0] Generating PoRep for sector (2)
beignet-opencl-icd: no supported GPU found, this is probably the wrong opencl-icd package for this hardware
(If you have multiple ICDs installed and OpenCL works, you can ignore this message)
beignet-opencl-icd: no supported GPU found, this is probably the wrong opencl-icd package for this hardware
(If you have multiple ICDs installed and OpenCL works, you can ignore this message)
2021-01-27T16:36:26.781+0800 INFO lotus-bench lotus-bench/main.go:660 [0] Unsealing sector
2021-01-27T16:36:26.783+0800 WARN ffiwrapper ffiwrapper/partialfile.go:70 could not allocated space, ignoring: operation not supported

The latest official Lotus used version is 0.12.5. Is it possible to support the new features on a 2080Ti graphics card

intel 4214 + 2080Ti run test got an error

git clone
git checkout 2080Ti
BELLMAN_VERIFIER=gpu BELLMAN_CUSTOM_GPU="GeForce RTX 2080 Ti:4352" BELLMAN_CPU_UTILIZATION=0 CUDA_VISIBLE_DEVICES=0 RUST_LOG=trace cargo test --features gpu -- --exact multiexp::gpu_multiexp_consistency --nocapture

warning: unused import: `std::time::Instant`
  --> src/gpu/multiexp.rs:14:5
   |
14 | use std::time::Instant;
   |     ^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_imports)]` on by default

warning: unused import: `std::time::Instant`
  --> src/gpu/multiexp.rs:14:5
   |
14 | use std::time::Instant;
   |     ^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_imports)]` on by default

warning: 1 warning emitted

warning: 1 warning emitted

    Finished test [unoptimized + debuginfo] target(s) in 0.08s
     Running target/debug/deps/bellperson-c7fae5472aad3398

running 1 test
[2021-02-03T09:43:33Z INFO  bellperson::gpu::utils] Device: Device { brand: Nvidia, name: "GeForce RTX 2080 Ti", memory: 11554717696, bus_id: 139, platform: Platform(PlatformId(0x7f850c025d70)), device: Device(DeviceId(0x7f850c032050)) }
Testing Multiexp for 1024 elements...
[2021-02-03T09:43:40Z INFO  bellperson::gpu::locks] GPU is available for Multiexp!
[2021-02-03T09:43:40Z DEBUG bellperson::gpu::locks] Acquiring GPU lock...
[2021-02-03T09:43:40Z DEBUG bellperson::gpu::locks] GPU lock acquired!
[2021-02-03T09:43:40Z INFO  bellperson::gpu::utils] Adding "GeForce RTX 2080 Ti" to GPU list with 4352 CUDA cores.
thread 'multiexp::gpu_multiexp_consistency' panicked at 'attempt to subtract with overflow', src/gpu/multiexp.rs:95:5
stack backtrace:
   0:     0x5630cbfefd45 - backtrace::backtrace::libunwind::trace::h14d338b30b3ea0a7
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1:     0x5630cbfefd45 - backtrace::backtrace::trace_unsynchronized::h73ea91d74a3fd67f
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2:     0x5630cbfefd45 - std::sys_common::backtrace::_print_fmt::hd42948c952866e12
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x5630cbfefd45 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha8f928866ff7571e
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x5630cc019c5c - core::fmt::write::he0c1e5f7426d2718
                               at src/libcore/fmt/mod.rs:1076
   5:     0x5630cbfec012 - std::io::Write::write_fmt::hf3afc6cfd57d0033
                               at src/libstd/io/mod.rs:1537
   6:     0x5630cbff24b0 - std::sys_common::backtrace::_print::hfc0110703f3696fd
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x5630cbff24b0 - std::sys_common::backtrace::print::h3f77c6990ddfaa22
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x5630cbff24b0 - std::panicking::default_hook::{{closure}}::heae49580a8d62d75
                               at src/libstd/panicking.rs:198
   9:     0x5630cbff21fc - std::panicking::default_hook::hecc34e3f729e213c
                               at src/libstd/panicking.rs:217
  10:     0x5630cbff2af3 - std::panicking::rust_panic_with_hook::he82f5d0644692441
                               at src/libstd/panicking.rs:526
  11:     0x5630cbff26eb - rust_begin_unwind
                               at src/libstd/panicking.rs:437
  12:     0x5630cc017bc1 - core::panicking::panic_fmt::h09c929f06bb87c98
                               at src/libcore/panicking.rs:85
  13:     0x5630cc017b0d - core::panicking::panic::h7ece43057e5422d4
                               at src/libcore/panicking.rs:50
  14:     0x5630cb97c11a - bellperson::gpu::multiexp::calc_chunk_size::hb8f64b88ca686bf8
                               at src/gpu/multiexp.rs:95
  15:     0x5630cb97c632 - bellperson::gpu::multiexp::SingleMultiexpKernel<E>::create::h2977a7ca19f6e7ab
                               at src/gpu/multiexp.rs:114
  16:     0x5630cb9831af - bellperson::gpu::multiexp::MultiexpKernel<E>::create::{{closure}}::h76dedb353ddec8e1
                               at src/gpu/multiexp.rs:244
  17:     0x5630cb882522 - core::iter::adapters::map_try_fold::{{closure}}::hd5753d2a861eaa39
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/iter/adapters/mod.rs:840
  18:     0x5630cb613481 - core::iter::traits::iterator::Iterator::try_fold::h803d94d9403c82a2
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/iter/traits/iterator.rs:1889
  19:     0x5630cb8a3484 - <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold::hdd1c383149ffd088
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/iter/adapters/mod.rs:866
  20:     0x5630cb87a758 - core::iter::traits::iterator::Iterator::find_map::h8eece059959af16d
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/iter/traits/iterator.rs:2264
  21:     0x5630cb864135 - <core::iter::adapters::FilterMap<I,F> as core::iter::traits::iterator::Iterator>::next::hf88fb15d7d13d56f
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/iter/adapters/mod.rs:1132
  22:     0x5630cb65852e - <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter::hf55def40768cf2f5
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/vec.rs:2093
  23:     0x5630cb665347 - <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter::h10e5a1bfcdedb730
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/vec.rs:1995
  24:     0x5630cb87914a - core::iter::traits::iterator::Iterator::collect::h7c4fa02120544595
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/iter/traits/iterator.rs:1671
  25:     0x5630cb981cf4 - bellperson::gpu::multiexp::MultiexpKernel<E>::create::h6c23d493f360f308
                               at src/gpu/multiexp.rs:242
  26:     0x5630cb54c910 - bellperson::multiexp::create_multiexp_kernel::hfd943f129225c659
                               at src/multiexp.rs:478
  27:     0x5630cb912799 - bellperson::gpu::locks::LockedMultiexpKernel<E>::init::h6850c33c191b55d4
                               at src/gpu/locks.rs:102
  28:     0x5630cb913048 - bellperson::gpu::locks::LockedMultiexpKernel<E>::with::h175adb60ef679ecf
                               at src/gpu/locks.rs:123
  29:     0x5630cb546f15 - bellperson::multiexp::multiexp::h21e9e52f6079f8f2
                               at src/multiexp.rs:327
  30:     0x5630cb5ca2cb - bellperson::multiexp::gpu_multiexp_consistency::h45ef679fd6884f8b
                               at src/multiexp.rs:526
  31:     0x5630cb54d2ba - bellperson::multiexp::gpu_multiexp_consistency::{{closure}}::h9f3da87cb803c9ad
                               at src/multiexp.rs:492
  32:     0x5630cb553aee - core::ops::function::FnOnce::call_once::hc4703ce24ef35065
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/ops/function.rs:233
  33:     0x5630cbd2441c - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h6633cb15d0d76942
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/boxed.rs:1081
  34:     0x5630cbd2441c - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h1e20120def172c5c
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:318
  35:     0x5630cbd2441c - std::panicking::try::do_call::hcc2ec3bbb75e9316
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:348
  36:     0x5630cbd2441c - std::panicking::try::h94eaebaaa7dd6f41
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:325
  37:     0x5630cbd2441c - std::panic::catch_unwind::h151c07c08497cf8b
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:394
  38:     0x5630cbd2441c - test::run_test_in_process::hd082de93b1922c89
                               at src/libtest/lib.rs:541
  39:     0x5630cbd2441c - test::run_test::run_test_inner::{{closure}}::h22369c9424e5ab3a
                               at src/libtest/lib.rs:450
  40:     0x5630cbcfc396 - std::sys_common::backtrace::__rust_begin_short_backtrace::h0660a89f67243e05
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/sys_common/backtrace.rs:130
  41:     0x5630cbd01445 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::h8bb2049509aa1add
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/thread/mod.rs:475
  42:     0x5630cbd01445 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb49f3484cb9c3dd3
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:318
  43:     0x5630cbd01445 - std::panicking::try::do_call::ha910a12d1577339b
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:348
  44:     0x5630cbd01445 - std::panicking::try::hd8b3d620360e55fa
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:325
  45:     0x5630cbd01445 - std::panic::catch_unwind::h475454730ea43154
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:394
  46:     0x5630cbd01445 - std::thread::Builder::spawn_unchecked::{{closure}}::h2407d9379d805151
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/thread/mod.rs:474
  47:     0x5630cbd01445 - core::ops::function::FnOnce::call_once{{vtable.shim}}::haef9f772dfab225e
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/ops/function.rs:233
  48:     0x5630cbff8f6a - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::hd2b3bc04af94a84f
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/boxed.rs:1081
  49:     0x5630cbff8f6a - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h1044417e186e567a
                               at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/boxed.rs:1081
  50:     0x5630cbff8f6a - std::sys::unix::thread::Thread::new::thread_start::h276e6ca033938925
                               at src/libstd/sys/unix/thread.rs:87
  51:     0x7f8514374609 - start_thread
  52:     0x7f8514280293 - __clone
  53:                0x0 - <unknown>
[2021-02-03T09:43:40Z DEBUG bellperson::gpu::locks] GPU lock released!
test multiexp::gpu_multiexp_consistency ... FAILED

failures:

failures:
    multiexp::gpu_multiexp_consistency

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 68 filtered out

error: test failed, to rerun pass '--lib'

关于 jack_chunk_3090 = 67108864; 67108864 如何计算出来的。

3090显卡下的显存溢出问题

2021-02-17T10:46:30.981 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.981 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:30.983 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.983 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.983 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:30.984 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.984 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.984 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:30.985 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.985 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.985 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:30.988 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:30.988 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:30.988 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:30.989 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:31.003 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:31.003 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.009 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.009 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.018 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.018 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.018 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.022 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.022 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.023 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.023 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.038 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.038 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.038 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.039 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.042 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.043 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.043 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.043 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.043 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.047 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.052 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.186 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-17T10:46:32.186 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-17T10:46:32.186 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-17T10:46:32.187 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.189 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.194 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:48:13.180 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:48:26.287 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:49:06.830 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:49:07.753 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:49:11.383 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:49:16.135 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:49:50.547 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:50:19.000 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:50:29.988 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:50:42.000 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:50:50.070 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:51:03.560 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:51:09.410 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:51:10.269 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:51:20.822 DEBUG bellperson::gpu::locks > GPU lock released!
2021-02-17T10:51:21.309 INFO bellperson::groth16::prover > ZQ: a_s end: 290.476105354s
2021-02-17T10:51:21.309 INFO bellperson::groth16::prover > ZQ: h_s start
2021-02-17T10:51:21.309 INFO bellperson::gpu::locks > GPU is available for Multiexp!
2021-02-17T10:51:21.309 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-02-17T10:51:21.309 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-02-17T10:51:21.310 INFO bellperson::gpu::utils > Adding "GeForce GTX 3090" to GPU list with 10496 CUDA cores.
2021-02-17T10:51:21.489 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0.2)
2021-02-17T10:51:21.490 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 3090 (Chunk-size: 67108864)
2021-02-17T10:51:21.490 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
2021-02-17T10:51:21.491 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:51:21.491 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-17T10:51:26.663 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:51:26.663 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte
2021-02-17T10:51:51.413 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:51:51.413 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-17T10:51:56.588 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:51:56.588 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte
2021-02-17T10:52:19.467 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:52:19.467 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-17T10:52:24.532 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:52:24.532 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte
2021-02-17T10:52:47.414 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:52:47.414 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-17T10:52:52.504 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:52:52.504 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte
2021-02-17T10:53:16.770 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:53:16.770 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-17T10:53:21.956 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:53:21.956 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte
2021-02-17T10:53:45.000 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:53:45.000 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-17T10:53:50.398 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:53:50.398 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte
2021-02-17T10:54:12.697 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:54:12.697 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-17T10:54:17.795 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144
2021-02-17T10:54:17.795 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte

EPYC 7302P + 3080 works

Tried /3080 branch.
Worked fine and finished C2 in 14 minutes.

2080ti，显存爆掉问题

p2做完会有300m的内存遗留，这样再做c2时候，+10.8g，会超出11g，导致-4报错。

只要是2080TI芯片的就可以吧，影驰华硕的2080显卡都可以挖矿是吧。

只要是2080TI芯片的就可以吧，影驰华硕的2080显卡都可以挖矿是吧。

EPYC 7302P + 3090 2 x C2 in parallel won't work

[](2021-02-08T00:08:47.967 INFO bellperson::groth16::prover > ZQ: build provers  end: 143.060379629s
2021-02-08T00:08:47.968 INFO bellperson::groth16::prover > ZQ: starting proof timer
2021-02-08T00:08:47.968 INFO bellperson::groth16::prover > ZQ: get params start
2021-02-08T00:10:20.109 INFO bellperson::groth16::prover > ZQ: waiting params...
2021-02-08T00:10:20.119 INFO bellperson::groth16::prover > ZQ: get params end: 92.15143134s
2021-02-08T00:10:20.119 INFO bellperson::groth16::prover > ZQ: a_s start
2021-02-08T00:10:20.372 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-02-08T00:10:21.291 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-02-08T00:10:21.291 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090
2021-02-08T00:10:21.291 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-02-08T00:20:51.937 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error: 

################################ OPENCL ERROR ############################### 

Error executing function: clEnqueueWriteBuffer  

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)  

Please visit the following url for more information: 

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors  

############################################################################# 

2021-02-08T00:24:33.141 INFO bellperson::groth16::prover > ZQ: a_s end: 853.022306209s
2021-02-08T00:24:33.141 INFO bellperson::groth16::prover > ZQ: h_s start
2021-02-08T00:24:33.142 INFO bellperson::gpu::locks > GPU is available for Multiexp!
2021-02-08T00:24:33.236 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
2021-02-08T00:24:33.236 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 3090 (Chunk-size: 67108864)
2021-02-08T00:24:33.236 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
2021-02-08T00:24:33.237 INFO bellperson::gpu::multiexp > GABEDEBUG: <G> size:104, <PrimeField> size:32, <Projective> size:144
2021-02-08T00:24:33.237 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte
2021-02-08T00:24:38.932 INFO bellperson::gpu::multiexp > GABEDEBUG: <G> size:104, <PrimeField> size:32, <Projective> size:144
2021-02-08T00:24:38.932 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413624byte, 20514Mbyte
2021-02-08T00:24:40.227 WARN bellperson::gpu::locks > GPU Multiexp failed! Falling back to CPU... Error: OpenCL Error: Ocl Error: 

################################ OPENCL ERROR ############################### 

Error executing function: clEnqueueWriteBuffer  

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)  

Please visit the following url for more information: 

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors  

############################################################################# 
****)

On official code, the same server can finish 2 C2 tasks in 40 minutes.
On the modified code here, it throws CL_MEM_OBJECT_ALLOCATION_FAILURE.

lotus-worker / 3090 Failed

I tested 3090 using lotus-bench, got the resault 9m21s.
Then I want to test it using lotus-worker. Everything seems going well, but got a ERR at last:
sector 0 got error event sealing.SectorSealPreCommit1Failed: consecutive invalid proofs
Im sure the c2 has done, but something like verifying failed at last.

2080Ti run lotus-bench error(not use gpu)

2021-01-26T00:31:58.945 INFO bellperson::gpu::locks > GPU is available for FFT!
2021-01-26T00:31:58.945 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-01-26T00:31:58.945 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-01-26T00:32:34.278 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2021-01-26T00:32:34.278 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 2080 Ti
2021-01-26T00:32:34.278 INFO bellperson::domain > GPU FFT kernel instantiated!
2021-01-26T00:36:38.486 INFO bellperson::groth16::prover > ZQ: a_s end: 279.617684661s
2021-01-26T00:36:38.527 DEBUG bellperson::gpu::locks > GPU lock released!
2021-01-26T00:36:38.527 INFO bellperson::groth16::prover > ZQ: h_s start
2021-01-26T00:36:38.527 INFO bellperson::gpu::locks > GPU is available for Multiexp!
2021-01-26T00:36:38.527 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2021-01-26T00:36:38.527 DEBUG bellperson::gpu::locks > GPU lock acquired!
2021-01-26T00:36:38.753 INFO bellperson::gpu::multiexp > Multiexp: 2 working device(s) selected. (CPU utilization: 0)
2021-01-26T00:36:38.753 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 44785859)
2021-01-26T00:36:38.753 INFO bellperson::gpu::multiexp > Multiexp: Device 1: GeForce RTX 2080 Ti (Chunk-size: 44785859)
2021-01-26T00:36:38.753 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
2021-01-26T00:36:57.762 WARN bellperson::gpu::locks > GPU Multiexp failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################