mimblewimble/cuckoo-miner

Mining wrapper around the Cuckoo Cycle proof of work system for the Grin/MimbleWimble blockchain.

cuckoo-miner's Issues

Use atomic bool instead of locking the structure

See #18 for reference.

Rust dynamic dependencies

It seems when building the plugins within cuckoo miner, cargo adds some dynamic dependencies to rust DLLs, causing loading to fail if they're not on the dynamic linker path. You don't see this issue in cargo, because cargo sets all the environment variables properly, but when running standalone, you might get a message that the plugin directory isn't able to load.

This may propagate into grin... at the moment there are two solutions:

Either add the rust DLLs to the linker path, e.g:

export LD_LIBRARY_PATH=/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib

OR

Build via the plugins via the cmake project, which is at:

https://github.com/mimblewimble/cuckoo-miner-plugins

When building via cmake, these dependencies aren't included, and the DLLs don't have the rust dependency.

I think the long term fix here is to include and use the cmake build instead of building the plugins internally, which will have benefits in that platform can be detected, the presence of CUDA/opencl can be detected and used if present and the right versions of the libraries can be built, etc. The downside is that anything that includes cuckoo-miner (just thinking of grin at the moment,) will need to have cmake installed as a build dependency, and it will take a little while to iron out the cmake build to ensure it works on all platforms.

Update parameters to support per GPU configuration

At the moment, running in async mode just tries to use all available GPUs.

We should be able to set individual parameters per GPU, as well as specify which GPUs should be used for mining.

Stratum/Pooled Mining implementation

Slightly later down the line, but it would nice to have a laying sitting on top of cuckoo-miner to deal with a stratum request/response workflow, to make pooled mining simple to integrate. Also have a look at the protocol to see what special considerations are needed for cuckoo.

MacOS pthread barrier implementation in cuckoo

When running mean miner compiled with NSIPHASH=1, XBITS=0 and without the maxv2 flag, there is a very occasional thread deadlock that appears to occur within the OSX pthread barrier implementation in @tromp's repository, and manifests as the miner appearing to stop within grin. The issue can be reproduced consistently on macOS if the original osx_barrier.h implementation is used, and is fixed when including a different implementation (currently in the master).

For now, I've just updated the mimblewimble/cuckoo-miner fork to use a different pthread barrier implementation, and it resolves the issue. However it would be better to get to the root cause of the issue, figure out how to reproduce it consistently, and change the upstream implementation.

Change default plugin output to reflect cuckoo plans

Right now several version of plugins are being output at various sizes. From discussions, the cuckoo implementations currently being planned are:

'lean' and 'mean' CPU implementations (currently cuckoo_miner.cpp and matrix_miner.cpp) in John Tromp's repository
'lean' and 'mean' GPU implementations(currently cuda_miner.cu and an upcoming GPU version of matrix_miner.cpp)

These will all be output at size 30, with 16 for grin's "UserTesting" mode as a nice-to-have. We may want to have a few different versions of the 'mean' CPU miner, as some instructions used within it only work on newer processors. Any parameters that aren't configurable at runtime should probably have a couple of default variations.

cuckoo_miner not compiling on Raspberry Pi 3 Jessie/GCC 6.3

Fur tracking purposes.

On a standard Raspberry Pi 3 (arch: armv7) where I "up-fitted" Jessie with GCC 6.3.0 (I can't use Stretch because of its wifi issues). Probable cause: hard-coded x86 compiler flags (-m64 -mavx2) in some build scripts.

Full build log here:
https://gist.github.com/oriste/6890a1d1b820704c036fb6d33d7a7c3e

Miner crashes with illegal instruction error on Atom CPU

I'm running on a server with an Atom C2550 CPU. When running grin with grin server --mine run, it crashes when it tries to start mining with:

Nov 19 05:35:25.588 INFO (Server ID: Port 13414) Starting miner loop.
Illegal instruction

If I leave off --mine, then it's fine, although of course it doesn't mine.

Let me know if theres a way that I can provide a backtrace from the mining plugin, or more information to help debug.

CMake Build Parameters

Add and document user configurable cmake parameters that will get passed through to the plugin builds for the different compile options on the plugins.

"Cuckoo30 at 0 gps (graphs per second)" when using cuda and USE_DEVICE = 0

Apparently the plugin doesn't work if in the grin-miner.toml
USE_DEVICE = 0
not commented, even tough ls /dev/nvidia* shows the device as 0th.

The default settings work but not if set explicity.

rust 1.28, grin-miner:master, cuda 9.2, GTX 1080 8GB

Unit Tests + CI Environment

Unit tests for everything:

-Testing output solutions against known solutions at various sizes, that can be run against new plugin implementations to test their outputs
-Load testing
-Plugin loading/unloading
-White box stress testing on the plugins themselves, to check for leaks and segfaults

Note there is some oddness with cargo testing in that loading and unloading DLLs seems to cause segfaults on exit with no obvious reason. We may perhaps have to use another testing strategy or here.

Crash in the cargo test

after issuing cargo test --features=build-cuda-plugins -- --nocapture

Thread 15 "on_cuda_commit_" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffeb5fd700 (LWP 1017)]
0x00007fffecd90dda in cuckoo_internal_process_hash (hash=0x7fffeb5fcdd8 "\265\026\365\006:\206o\246\300\262\311\275\265\306I\357F\\B \205\342\231\357<\v\334%\"x\230\225", hash_length=32,
    nonce=0x7fffeb5fcdd0 "\376Y\r\004\217^<d\265\026\365\006:\206o\246\300\262\311\275\265\306I\357F\\B \205\342\231\357<\v\334%\"x\230\225") at /usr/src/cuckoo-miner/src/cuckoo_sys/plugins/cuckoo/src/cuckoo_miner/cuda_miner_adds.h:305
305	    DEVICE_INFO[device_id].is_busy=true;
(gdb) bt
#0  0x00007fffecd90dda in cuckoo_internal_process_hash (hash=0x7fffeb5fcdd8 "\265\026\365\006:\206o\246\300\262\311\275\265\306I\357F\\B \205\342\231\357<\v\334%\"x\230\225", hash_length=32,
    nonce=0x7fffeb5fcdd0 "\376Y\r\004\217^<d\265\026\365\006:\206o\246\300\262\311\275\265\306I\357F\\B \205\342\231\357<\v\334%\"x\230\225") at /usr/src/cuckoo-miner/src/cuckoo_sys/plugins/cuckoo/src/cuckoo_miner/cuda_miner_adds.h:305
#1  0x00007fffecd8fd72 in cuckoo_process (args=0x0) at /usr/src/cuckoo-miner/src/cuckoo_sys/plugins/cuckoo/src/cuckoo_miner/cuckoo_miner.h:264

'Halting' Plugins

At the moment, there's a need to stop cuckoo processing if a solution's already been found, figure out where the best place to put this is in the various cuckoo-miner plugin implementations so that it doesn't impact on solution times.

Returning multiple solutions, if found

Currently only returning the first solution found for a hash. Extend API and plugin modifications to handle returning all found solutions from a hash.

Support Simultaneous CPU+GPU Mining

At the moment, cuckoo-miner only supports loading a single plugin at a time. Extend to cover an arbitrary number of plugins loaded at once, to allow for simultaneous GPU and CPU mining.

When mining plugins do not build, this does not generate a build error

I tried building grin-miner on Ubuntu 18.04.1 on WSL. This appears to work, with a message "Compiling cuckoo_miner v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_34#c2988598)" during build and no errors. However, the target/debug/plugins directory is missing plugins, although cuckoo_miner is present in target/debug/build.

I'd expect a build-time error message when plugins fail to build.

This appears related to the Windows detection in build.rs. Although WSL is an Ubuntu environment and should be treated as Linux, the $PATH contains the Windows %PATH% as well, which means Finder will find .exe files.

Fails on build on windows

Trying to build on x86_64-pc-windows-msvc result with:

cargo check
...
Compiling env_logger v0.3.5
Compiling cuckoo_edgetrim_16 v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_2#43fd4177)
Compiling cuckoo_tomato_25 v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_2#43fd4177)
Compiling cuckoo_simple_12 v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_2#43fd4177)
Compiling cuckoo_tomato_16 v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_2#43fd4177)
Compiling cuckoo_tomato_30 v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_2#43fd4177)
Compiling cuckoo_simple_16 v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_2#43fd4177)

...

error: failed to run custom build command for `cuckoo_tomato_16 v0.1.0 (https://github.com/mimblewimble/cuckoo-miner?tag=grin_integration_2#43fd4177)`
process didn't exit successfully: `C:\Users\Latrasis\Documents\Repos\grin\target\debug\build\cuckoo_tomato_16-758c64c2c02ca080\build-script-build` (exit code
: 101)

...

running: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\VC\\Tools\\MSVC\\14.10.25017\\bin\\HostX64\\x64\\cl.exe" "/nologo" "/MD" "/Z7
" "/I" "../../cuckoo/src" "/I" "../../cuckoo/src/cuckoo_miner" "/I" "." "-EHsc" "-DEDGEBITS=27" "/DNDEBUG=1" "/FoC:\\Users\\Raketa\\Documents\\Repos\\grin\
\target\\debug\\build\\cuckoo_simple_28-ea166400c846717c\\out/cuckoo-28\\../../cuckoo/src\\simple_miner.o" "/c" "../../cuckoo/src/simple_miner.cpp"
simple_miner.cpp
c:\users\latrasis\.cargo\git\checkouts\cuckoo-miner-4752934f0f1f2bfe\43fd417\cuckoo-sys\cuckoo\src\siphash.h(6): fatal error C1083: Cannot open include file:
 'endian.h': No such file or directory
exit code: 2

bug in meets_difficulty

Looks to me like > should be >=

This would explain why the grin explorer only shows solutions whose difficulty strictly exceeds the minimum required. Everybody is mining at inflated difficulty.