helium/miner

Miner for the helium blockchain

miner's Issues

Update staking keys for Syncrobit

ADD new maker (Syncrob.it):
14rb2UcfS9U89QmKswpZpjRCUVCVu1haSyqyGY486EvsYtvdJmR

Does miner take a long time to restart?

I installed the miner on AWS. When I issue the command docker exec miner miner info height I get the following response:
Error response from daemon: Container XXXXXXXXXXXXXXXXXXX is restarting, wait until the container is running

Its been several hours already

Missing LoraWAN support code frequency for my country, Uganda.

According to the information provided by Lora alliance org on the list of countries with supported frequencies for LoraWan My country (Uganda) is among and lies within this range of frequency; 923 - 925 MHz plus 863 - 865 MHz,
865-867.6 MHz, 869.25 - 869.7 MHz; with these support codes (EU433 and AS923). I would like to start mining Helium (HNT) in Uganda However I'm not sure whether the frequencies in Uganda will be compatible with the LoRaWAN devices. The Github list does not show whether Uganda frequency ranges support the Helium network in Uganda in case I buy/order/ship my the devices to Uganda and connect them will they work? Plus I see that my country shares the same codes on the Lora alliance org with many other countries like Hong Kong, UK, Thailand, that are currently mining Helium but on your list Uganda's information is not matching that on Lora alliance org. Could you please crosscheck for me the frequency to confirm if I can mine Helium in Uganda using the current mining LoRaWAN devices within those radio frequency ranges in Uganda as per Lora alliance organization list of supported countries.

Log to stdout/stderr?

It looks like the miner is hardcoded to use console.log and error.log as its logging output. When running in a docker container it would be nice to be able to set stdout and stderr as the logging output, and handle log file management upstream of the container.

Fix build status

Hard to tell if current code is ready for deployment when build fail.

start sibyl and grpc services when in validator mode only

Sibly app will be pulled into miner as an included app. We will only want to start it if the miner is running in validator mode. It should not run when in gateway mode.

Add a multi-arch latest tag

In the spirit of 7e0f4da, it would be nice if you pushed a multi-arch latest tag, so that one could just docker pull quay.io/team-helium/miner instead of having to docker pull quay.io/team-helium/miner:latest-amd64 (or arm64). This would also make it easier for newcomers and scripts.

I don't know Buildkite, but it should be as simple as:

docker manifest create \
  quay.io/team-helium/miner:latest \
  --amend quay.io/team-helium/miner:latest-amd64 \
  --amend quay.io/team-helium/miner:latest-arm64
docker push quay.io/team-helium/miner:latest

as a last step in your pipeline (or with buildx).

Also, do you have anything planned for hacktoberfest? 😜

Consensus group and DKG telemetry

This is a bit intentionally vague; we don't, in the end, know what this is going to look like.

It would be very desirable to have members of the consensus group and new candidates report fine-grained progress statistics so that we can better understand what's going wrong during long blocks and halts, as well as gather information about where it would be profitable to optimize further.

Validators - allow restaking a validator in cool down

Per the description in HIP 25, it was intended that a stake in cool down could be re-staked. This functionality does not seem to be implemented. Attempting to stake a validator that is in cool down returns a {invalid,already_exists} failure.

unstake_validator: validator pubkey: this starts the unlock process for the stake, returning it to the owner's account after some cooldown period. the validator is immediately unable to join block production groups, but re-staking will restore the validator to eligibility.

Update: I tested this by creating a stake transaction on a wallet which has a validator in cool down. I tried this both with the same validator address that was in cool down as well as a different address that had never been staked. In the first case, the transaction failed with already_exists and in the second case it failed because of insufficient balance.

Bump deps

The following deps have build-system improvements for macOS builds and OTP23 support

• https://github.com/helium/erlang-pbc
• https://github.com/helium/erlang-erasure
• https://github.com/helium/ebus

These deps have outstanding issues:

• https://gitlab.com/barrel-db/erlang-rocksdb/-/issues/118

cannot find -lerl_interface

Trying to install this on my raspberry pi, not sure if it´s possible though... getting some errors at the end of make release

make[1]: Entering directory '/home/pi/github/miner/_build/default/lib/erlang_ale/c_src' cc -O3 -finline-functions -Wall -Wextra -Wno-unused-parameter -I /usr/local/lib/erlang/erts-11.1.4/include/ -I /usr/local/lib/erlang/lib/erl_interface-4.0.1/include -c -o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/gpio_port.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/gpio_port.c cc -O3 -finline-functions -Wall -Wextra -Wno-unused-parameter -I /usr/local/lib/erlang/erts-11.1.4/include/ -I /usr/local/lib/erlang/lib/erl_interface-4.0.1/include -c -o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/erlcmd.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/erlcmd.c cc -O3 -finline-functions -Wall -Wextra -Wno-unused-parameter -I /usr/local/lib/erlang/erts-11.1.4/include/ -I /usr/local/lib/erlang/lib/erl_interface-4.0.1/include -c -o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/spi_port.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/spi_port.c cc -O3 -finline-functions -Wall -Wextra -Wno-unused-parameter -I /usr/local/lib/erlang/erts-11.1.4/include/ -I /usr/local/lib/erlang/lib/erl_interface-4.0.1/include -c -o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/ale_main.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/ale_main.c cc -O3 -finline-functions -Wall -Wextra -Wno-unused-parameter -I /usr/local/lib/erlang/erts-11.1.4/include/ -I /usr/local/lib/erlang/lib/erl_interface-4.0.1/include -c -o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/i2c_port.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/i2c_port.c cc /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/gpio_port.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/erlcmd.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/spi_port.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/ale_main.o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/i2c_port.o -L /usr/local/lib/erlang/lib/erl_interface-4.0.1/lib -lerl_interface -lei -o /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/../priv/erlang-ale /usr/bin/ld: cannot find -lerl_interface collect2: error: ld returned 1 exit status make[1]: *** [Makefile:46: /home/pi/github/miner/_build/default/lib/erlang_ale/c_src/../priv/erlang-ale] Error 1 make[1]: Leaving directory '/home/pi/github/miner/_build/default/lib/erlang_ale/c_src' ===> Hook for compile failed! make: *** [Makefile:32: release] Error 1

Miner version miner-amd64_2020.xx.yy.z_GA in docker crashes when elected for consensus group

Miner version miner-amd64_2020.10.31.0_GA when running in docker crashes when elected for consensus group.

The miner was Elected for consensus at 11:20 pm October 30. After that moment PoC stops and only traffic is registered. at 3:12 am October 31, traffic stops and the miner starts crashing. In the Helium app the miner was reported syncing. When inspecting the docker container it was found that it kept crashing.

removing the minder_data directory, rebuilding it, starting sync the blockchain and importing a snapshot form another miner recovered brought the miner to normal operation.

docker build failing

sorry this is all I have. I was hoping this would help

2%] Building CXX object CMakeFiles/rocksdb.dir/db/compaction/compaction.cc.o
[ 4%] Building CXX object CMakeFiles/rocksdb.dir/db/compaction/compaction_iterator.cc.o
[ 4%] Building CXX object CMakeFiles/rocksdb.dir/db/compaction/compaction_picker.cc.o
[ 4%] Building CXX object CMakeFiles/rocksdb.dir/db/compaction/compaction_job.cc.o
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[6]: *** [CMakeFiles/rocksdb.dir/build.make:193: CMakeFiles/rocksdb.dir/db/compaction/compaction_job.cc.o] Error 1
make[6]: *** Waiting for unfinished jobs....

make[5]: *** [CMakeFiles/Makefile2:142: CMakeFiles/rocksdb.dir/all] Error 2
make[4]: *** [CMakeFiles/Makefile2:154: CMakeFiles/rocksdb.dir/rule] Error 2
make[3]: *** [Makefile:190: rocksdb] Error 2
make[2]: *** [CMakeFiles/rocksdb.dir/build.make:112: rocksdb-prefix/src/rocksdb-stamp/rocksdb-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:80: CMakeFiles/rocksdb.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

• exit 1
  ===> Hook for compile failed!

The command '/bin/sh -c rebar3 as prod tar' returned a non-zero code: 1

I'm running this in a 64bit image on a raspbery pi 3.

libp2p_yamux_stream noproc errors

Hi,

I'm trying to run miner (on AMD64 in Docker latest-amd, sha256:bb883c6179406796dd5191cd6d2769c9cc98e278e8b2fa0c3d61a18035e15c2d) on a platform.

It seems to be working (blocks being received, communication with the packet-forwarder). But the error-log and crash-log are filled with these messages:
crash.log:

2021-03-05 06:46:55 =SUPERVISOR REPORT====
     Supervisor: {<0.9623.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.9626.3>},{id,1},{mfargs,{libp2p_yamux_stream,open_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:47:06 =SUPERVISOR REPORT====
     Supervisor: {<0.9650.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.9654.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:47:53 =SUPERVISOR REPORT====
     Supervisor: {<0.9730.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.9734.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:48:08 =SUPERVISOR REPORT====
     Supervisor: {<0.9822.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.9940.3>},{id,4},{mfargs,{libp2p_yamux_stream,open_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:49:02 =SUPERVISOR REPORT====
     Supervisor: {<0.9926.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.9930.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:49:52 =SUPERVISOR REPORT====
     Supervisor: {<0.10297.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10301.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:49:55 =SUPERVISOR REPORT====
     Supervisor: {<0.10325.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10330.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:50:23 =SUPERVISOR REPORT====
     Supervisor: {<0.10196.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10199.3>},{id,1},{mfargs,{libp2p_yamux_stream,open_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:50:24 =SUPERVISOR REPORT====
     Supervisor: {<0.10516.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10520.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:50:37 =SUPERVISOR REPORT====
     Supervisor: {<0.10658.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10662.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:50:42 =SUPERVISOR REPORT====
     Supervisor: {<0.10695.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10699.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:51:03 =SUPERVISOR REPORT====
     Supervisor: {<0.10869.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10873.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:51:29 =SUPERVISOR REPORT====
     Supervisor: {<0.10587.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.10595.3>},{id,1},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:52:42 =SUPERVISOR REPORT====
     Supervisor: {<0.2528.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.2535.3>},{id,3},{mfargs,{libp2p_yamux_stream,open_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:53:15 =SUPERVISOR REPORT====
     Supervisor: {<0.11098.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.11101.3>},{id,1},{mfargs,{libp2p_yamux_stream,open_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:56:00 =SUPERVISOR REPORT====
     Supervisor: {<0.11396.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.11399.3>},{id,1},{mfargs,{libp2p_yamux_stream,open_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-03-05 06:57:06 =SUPERVISOR REPORT====
     Supervisor: {<0.11530.3>,libp2p_simple_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.11533.3>},{id,1},{mfargs,{libp2p_yamux_stream,open_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

error.log:

2021-03-05 06:31:51.662 [error] <0.7495.3> Supervisor {<0.7495.3>,libp2p_simple_sup} had child 5 started with {libp2p_yamux_stream,open_stream,undefined} at <0.7574.3> exit with reason noproc in context shutdown_error
2021-03-05 06:33:32.861 [error] <0.7640.3> Supervisor {<0.7640.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.7643.3> exit with reason noproc in context shutdown_error
2021-03-05 06:34:04.663 [error] <0.7691.3> Supervisor {<0.7691.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.7694.3> exit with reason noproc in context shutdown_error
2021-03-05 06:34:39.784 [error] <0.7747.3> Supervisor {<0.7747.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.7750.3> exit with reason noproc in context shutdown_error
2021-03-05 06:35:53.156 [error] <0.7954.3> Supervisor {<0.7954.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.7958.3> exit with reason noproc in context shutdown_error
2021-03-05 06:36:53.522 [error] <0.8088.3> Supervisor {<0.8088.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.8092.3> exit with reason noproc in context shutdown_error
2021-03-05 06:40:08.015 [error] <0.8551.3> Supervisor {<0.8551.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.8554.3> exit with reason noproc in context shutdown_error
2021-03-05 06:40:10.990 [error] <0.8638.3> Supervisor {<0.8638.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.8642.3> exit with reason noproc in context shutdown_error
2021-03-05 06:40:14.345 [error] <0.8593.3> Supervisor {<0.8593.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.8596.3> exit with reason noproc in context shutdown_error
2021-03-05 06:46:55.745 [error] <0.9623.3> Supervisor {<0.9623.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.9626.3> exit with reason noproc in context shutdown_error
2021-03-05 06:47:06.274 [error] <0.9650.3> Supervisor {<0.9650.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.9654.3> exit with reason noproc in context shutdown_error
2021-03-05 06:47:53.611 [error] <0.9730.3> Supervisor {<0.9730.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.9734.3> exit with reason noproc in context shutdown_error
2021-03-05 06:48:08.452 [error] <0.9822.3> Supervisor {<0.9822.3>,libp2p_simple_sup} had child 4 started with {libp2p_yamux_stream,open_stream,undefined} at <0.9940.3> exit with reason noproc in context shutdown_error
2021-03-05 06:49:02.137 [error] <0.9926.3> Supervisor {<0.9926.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.9930.3> exit with reason noproc in context shutdown_error
2021-03-05 06:49:52.962 [error] <0.10297.3> Supervisor {<0.10297.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.10301.3> exit with reason noproc in context shutdown_error
2021-03-05 06:49:55.793 [error] <0.10325.3> Supervisor {<0.10325.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.10330.3> exit with reason noproc in context shutdown_error
2021-03-05 06:50:23.447 [error] <0.10196.3> Supervisor {<0.10196.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.10199.3> exit with reason noproc in context shutdown_error
2021-03-05 06:50:24.328 [error] <0.10516.3> Supervisor {<0.10516.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.10520.3> exit with reason noproc in context shutdown_error
2021-03-05 06:50:37.021 [error] <0.10658.3> Supervisor {<0.10658.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.10662.3> exit with reason noproc in context shutdown_error
2021-03-05 06:50:42.024 [error] <0.10695.3> Supervisor {<0.10695.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.10699.3> exit with reason noproc in context shutdown_error
2021-03-05 06:51:03.459 [error] <0.10869.3> Supervisor {<0.10869.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.10873.3> exit with reason noproc in context shutdown_error
2021-03-05 06:51:29.949 [error] <0.10587.3> Supervisor {<0.10587.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive_stream,undefined} at <0.10595.3> exit with reason noproc in context shutdown_error
2021-03-05 06:52:42.994 [error] <0.2528.3> Supervisor {<0.2528.3>,libp2p_simple_sup} had child 3 started with {libp2p_yamux_stream,open_stream,undefined} at <0.2535.3> exit with reason noproc in context shutdown_error
2021-03-05 06:53:15.163 [error] <0.11098.3> Supervisor {<0.11098.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.11101.3> exit with reason noproc in context shutdown_error
2021-03-05 06:56:00.018 [error] <0.11396.3> Supervisor {<0.11396.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.11399.3> exit with reason noproc in context shutdown_error
2021-03-05 06:57:06.875 [error] <0.11530.3> Supervisor {<0.11530.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stream,undefined} at <0.11533.3> exit with reason noproc in context shutdown_error

Erlang Install Fail (Wrong Architecture?)

As per: https://github.com/helium/miner#installing-miner-from-source

Instructions says errors after 'sudo dpkg -i esl-erlang_22.1.6-1raspbianbuster_armhf.deb' are expected but I got ...

dpkg: error processing archive esl-erlang_22.1.6-1raspbianbuster_armhf.deb (--install):
package architecture (armhf) does not match

sudo apt-get install -f returns;

Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Is erlang installed or not?

uname -a returns:

Linux Helium 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

So I'm on an Intel box?

Enhance run.sh to use multi-sig

Instead of generating a singular master key, we should change this code to optionally generate a set of multi-keys.

Validators - transfer stake transaction should include stake amount transferred

I believe that the transfer stake transaction should include the amount of stake being transferred.

First, with the potential for overstake at some point in the future, this may be required if partial transfer is also enabled. Adding the amount now would allow this transaction to be compatible with the potential future functionality and void a transfer v2 transaction.

Second and more important, this is needed from an accounting perspective for wallet owners. Without the stake amount included in the transfer transaction, the transaction history does not correspond to the change in stake balance for the account.

Put another way, it is not possible to derive the staked balance at a point in time by looking solely by looking at transactions for that account. Technically, this can be inferred from looking back at the original transaction for the amount. However, this adds a lot of complexity when it comes to accounting or tax reporting in order to be able to track account balance at any given time.

Macedonia Frequency

From the LoRaWAN Regional Parameters guidelines, it looks like Macedonia uses the EU863-870 frequency. Can this be checked to add it to the supported frequencies? https://lora-alliance.org/wp-content/uploads/2020/11/rp_2-1.0.1.pdf. Thanks.

Validator - miner peer book -s showing listen address that is not on device

Version: 0.1.20.

When checking status of with miner peer book -s, a second IP address that is not related to the validator is listed as a peer address.

|                   address                    |     name      |listen_add|connectio|nat|last_updat|
+----------------------------------------------+---------------+----------+---------+---+----------+
|/p2p/1Za3L1R32qPJJ546kteo7WP9t2DzWtTMw5u9jKfpJ|magic-caramel-s|    2     |    8    |non| 120.272s |
+----------------------------------------------+---------------+----------+---------+---+----------+

+----------------------------+
| listen_addrs (prioritized) |
+----------------------------+
|/ip4/158.69.118.209/tcp/2154|
|/ip4/75.119.137.119/tcp/2154|
+----------------------------+

In this example, the top IP address is of the validator where this command is being run. The lower IP address is a public IP address of a different validator completely.

Please move docker validator logs to the data dir

On val 0.1.8, docker image, the container ends with a sys.config that has
{log_root, "/var/log/miner"}

The miner docker container has:
{log_root, "/var/data/log"}
which puts the logs in the mounted data directory and outside the container.

Prepare balance of the test token (TNT) for the testnet chain

Provide initial balance of the test token (TNT) for the testnet chain. This token will be allocated to testers to use for staking their test validator node to testnet, and also as rewards for staking.

Update Staking Keys for Nebra

Remove the old Helium staking key 14fzfjFcHpDR1rTH8BNPvSi5dKBbgxaDnmsVPbCjuq9ENjpZbxh
Add NEW staking key (Nebra Ltd) 13Zni1he7KY9pUmkXMhEhTwfUpL9AcEV1m2UbbvFsrU9QPTMgE3

Miner should provide more node information for easier debugging and API access to data

I'm looking to give API users and ultimately Hotspot owners more access to information about their hotspot without having to interrogate it locally via Bluetooth.

Proposal is to share the following metadata:

• hardware version (rakspot, blockspot, sidetable_v1, sidetable_v2)
• firmware version
• uptime

I think these three metrics will be a nice path towards self service support and give the blockchain team (and community) visibility on what's running in the network.

Create a `validator miner info summary` for quick list of useful validator monitoring cmds

Rather than checking things individually like block height, in consensus, listen addr, peer book entries, build version, recent errors logged, etc. the validator miner info summary would pull all of these in one cmd

OTP 23 + rebar3 3.14

It's about time to start moving towards OTP 23, and rebar3 3.14 makes a lot of changes to how the scripts run, so we need to spend some time updating clique and our stuff to make all this work. Some of the work has already been done.

Certain Data Payloads in PushData frames crash Miner

Following reports from Disk91 on Discord, I came across something that seems quite brittle. "Bad payloads" seem to crash the Miner.

The payload "AA==" is enough to trigger a reboot loop:

2020-12-02 22:38:03.210 1 [info] <0.3620.1>@miner_lora:handle_udp_packet:366 PUSH_DATA [{<<"rxpk">>,[[{<<"chan">>,0},{<<"codr">>,<<"4/5">>},{<<"data">>,<<"AA=">>},{<<"datr">>,<<"SF8BW500">>},{<<"freq">>,902.8},{<<"lsnr">>,-15.0},{<<"modu">>,<<"Foo">>},{<<"rfch">>,0},{<<"rssi">>,-80},{<<"size">>,12},{<<"stat">>,12},{<<"tmst">>,12}]]}] from 67305985 on 57856
2020-12-02 22:38:03.210 1 [error] <0.3620.1>@base64:decode_binary:374 gen_server miner_lora terminated with reason: no function clause matching base64:decode_binary(<<>>, <<>>, 0, 0, eq) line 374
2020-12-02 22:38:03.210 1 [error] <0.3620.1>@base64:decode_binary:374 CRASH REPORT Process miner_lora with 0 neighbours crashed with reason: no function clause matching base64:decode_binary(<<>>, <<>>, 0, 0, eq) line 374
2020-12-02 22:38:03.211 1 [warning] <0.1210.0>@blockchain_event:terminate:108 terminating remove_handler

testnet: helium-wallet info, balance 10005, right after staking

Like others, I noticed that the initial balance in the wallet is 10005 (after staking) and should be zero? Potential for someone to unstake at that point and get 10005 TNT for doing nothing? concern?

Reject oversize txn batches from consensus members

When we have decoded a consensus group member's txn proposals, we should ignore them if there are too many.

Audit rescue blocks

Ideally someone would add to the audit directory information about all known rescue blocks, on the theory that anything done with the master key should be transparently documented, including links to incident reports, rationale, etc.

Known rescue blocks:
11578
22462, 22508 (same incident, I think)
121439

relx version used uses non portable code

relx contains a function to generate a random hash: relx_gen_id

The current version of relx contains a portable way of doing so:

dd count=1 bs=4 if=/dev/urandom 2> /dev/null | od -x  | head -n1 | awk '{print $2$3}'

However the version currently embedded with the miner does the following:

od -t x -N 4 /dev/urandom | head -n1 | awk '{print $2}'

The problem for this is that the busybox implementation of od doesn't support the arguments. Example on a Kerlink gateway:

root@klk-wiis-050029:~ # dd count=1 bs=4 if=/dev/urandom 2> /dev/null | od -x  | head -n1 | awk '{print $2$3}'
955ffdcc
root@klk-wiis-050029:~ # od -t x -N 4 /dev/urandom | head -n1 | awk '{print $2}'                              
od: invalid option -- 't'
BusyBox v1.24.1 (2020-08-03 15:13:50 CEST) multi-call binary.

Usage: od [-aBbcDdeFfHhIiLlOovXx] [FILE]

I'd have submitted a PR if I knew how relx is invoked.

Validators - allow transfer to new owner with same validator address

The transfer stake transaction should support transferring the stake to a new owner while maintaining the existing validator address. A stake owner may want to transfer their stake to a different owner without needing to change the validator. There is a straightforward workaround of regenerating the swarm_key and restarting the validator so this is a convenience rather than a must have.

Currently, a transfer stake transaction is considered invalid anytime the old address equals the new address. I believe the logic should be changed to also considered whether the new owner does not equal the old owner.

Update staking keys for Bobcat

ADD Bobcat Maker.

address 14sKWeeYWQWrBSnLGq79uRQqZyw3Ldi7oBdxbF6a54QboTNBXDL

can this work on a raspian rasperry pi

I tried to install it on rasperry pi running raspian and I get the following error.

pi@raspberrypi:/miner $ make
./rebar3 compile
./rebar3: error while loading shared libraries: /lib/arm-linux-gnueabihf/libtinfo.so.5: invalid ELF header
make: *** [Makefile:17: compile] Error 127
pi@raspberrypi:/miner $ sudo apt-get /lib/arm-linux-gnueabihf
E: Invalid operation /lib/arm-linux-gnueabihf

can anyone advice how to fix this?

Peer Book CLI Query Leaves Stranded Processes

Reported on Discord, but I have not confirmed

https://discordapp.com/channels/404106811252408320/730246424666832947/794296523298177066

Miner cli comand miner ledger gateways fails

The current miner command miner ledger gateways fails to do anything. It will throw the following error: RPC to '[email protected]' failed: timeout.

I suspect this is due to the shear amount of active gateways on the networks. I have tried this on 4 different docker containers on AWS t2.medium, and Digital Ocean smallest droplet, Linode 2nd cheapest plan, and on my personal computer. They all throw the same error. I believe it has to do something with the docker container and the hotspots being under powered.

Validators - allow transfer stake during cooldown

Given the long cooldown period proposed for stakes, there may be demand for the option to transfer a stake while it is in the cooldown period. Currently, as soon as a validator is unstaked, the owner is locked into waiting for the stake to complete the cooldown period. Owners may desire to have the flexibility to sell the cooling stake.

This may also require a new transaction type or new attribute of a transfer stake transaction to ensure that the new owner has certainty as to whether they are accepting an active versus cooling stake. If not, it may be used for malicious purposes. For example, if someone agrees to sell and transfer an active stake; they could initiate the transfer, send the partially signed transaction to the new owner for signing and submitting, but in the mean time unstake. The transfer would still go through although the new owner may be expecting it to be an active stake.

Validators - unstake transactions should include amount

I believe that the unstake transaction should include the amount that is being unstaked.

First, with the potential for overstake at some point in the future, this may be required if partial unstake is also enabled. Adding the amount now would allow this transaction to be compatible with the potential future functionality and void an unstake v2 transaction.

Second and more important, this is needed from an accounting perspective for wallet owners. Currently, there is no account activity record that corresponds to a stake being returned to a wallet. The 'stake' transaction includes an amount and the amount leads to a direct reduction in the available account balance decreasing. However, when unstaking, there is no corresponding transaction/activity (with an amount) that indicates the stake being returned to the account balance.

Put another way, it is not possible to derive the account balance by looking solely by looking at transactions for that account.
Technically, this can be derived from looking back at the original transaction for the amount as well as the cooldown period in the chain vars. However, this adds a lot of complexity when it comes to accounting or tax report in order to be able to track account balance at any given time.

Furthermore on the accounting point, it might even require adding another transaction type that is generated at the time that the cooldown period ends and the stake is put back in the account's available balance. There are technically three transition that occur but only two transaction types:

1. Not staked to Staked (the current stake transaction)
2. Staked to Cooldown (the current unstake transaction, although it doesn't include amount)
3. Cooldown to not staked (no corresponding transaction)

Tracking issue: Update RF acceptance curves

This is a meta issue for some cross cutting work to improve our RF acceptance criteria for PoC witnesses.

First, some graphs:

This is the current Free Space Path Loss (FSPL) acceptance curve (in red) against a scatter plot of distance (in meters) vs RSSI in the US915 region. As you can see it is extremely lenient. In yellow you see the curve adjusted for actual output power and finally in green you see the yellow curve with the RSSI values multipled by 1.3 to attempt to account for some real world losses.

Beyond this, we currently use the same curve in EU868 (and other regions), where it's even less applicable:

Clearly correcting for transmit power and frequency are important. We also need to come up with a reasonable loss factor, whether it's simply multiply by 1.3 or something better (arguably 1.4 might be a better fit).

Next, consider the RSSI acceptance curve. Again the current curve is in red. @JayKickliter has made direct observations of the radio by cabling the transmitter to the receiver and adding attenuation. These are the green and yellow lines. One thing of note is that SX1302 reports both channel RSSI as well as signal RSSI. The channel RSSI seems to bottom out at -117 but the signal RSSI can go lower. The miner should be corrected to report signal RSSI (RSSIS) when available.

Another thing to note is that the current curve (which was computed against different data) has a spot where it significantly goes above the observed minimum values. This is contributing to cases where we are improperly rejecting witness reports.

Again, here's the same thing in EU868, note that the current curve has almost no impact because it was computed for US915 and with different transmit options:

Also notice the values seemingly following the RSSIS curve at the bottom, we need to investigate where these are coming since we do not report RSSIS and we believe RSSI bottoms out at -117.

Here's an initial list of things we should do:

• Report RSSIS instead of RSSI when available.
• Determine loss factor for FSPL (and make it a chain variable). Potentially different loss factors for different regions.
• Compute FSPL per region using region specific TX power and Frequency.
• Try to curve-fit the observed RSSIS curves and replace the current acceptance curve with a per-region one.
• Investigate lower than expected possible RSSI reports in EU868 (which hotspots are reporting these values and why).
• Allow hotspots to declare their antenna configuration (tx/rx gain) and thread those declared values into the FSPL calculation. Tx power would be lowered for high-tx-gain antennas to preserve regulatory compliance.
• Build a geographic location to regional parameter lookup table. Right now the miner uses the API to determine this, but we need to have a proper lookup table. Should return a region code eg. US915 that we can then look up for parameters.
• Characterize other regions we support or plan to support soon. If we change regional parameters (eg the stronger EU868 poc channel proposal) characterize those as well.

Failed to dial peer and drops packet

Description

I'm using a DIY gateway with a miner hosted in an AWS EC2 instance to try to send data to the Helium network. Some packets are being dropped in between the miner and reaching the Helium console (I'd estimate that 1 out of every ~5-10 packets is dropped). I have tracked several dropped packets and they appear to move successfully from the sensor, through the packet forwarder, and to the miner, where it is dropped with the error messages failed to dial "/p2p/112qB3YaH5bZkCnKA5uRH7tBtGNv2Y5B4smv1jsmvGUzgKT71QpE" and failed to dial 1: failed dropping 2 packets.

I've included the logs from the miner during a dropped packet below. The payload being sent is 3 bytes (not including the Helium header) and is being sent every 2 minutes (for testing purposes). The issue also happens at higher intervals between packets (I know that I've seen the issue at 5 minute intervals as well).

2020-08-04 18:33:37.198 [info] <0.1321.0>@miner_lora:handle_udp_packet:380 PUSH_DATA [{<<"rxpk">>,[[{<<"tmst">>,676026522},{<<"time">>,<<"2020-08-04T18:33:36.169971Z">>},{<<"tmms">>,1280601235170},{<<"chan">>,8},{<<"rfch">>,0},{<<"freq">>
,904.6},{<<"stat">>,1},{<<"modu">>,<<"LORA">>},{<<"datr">>,<<"SF8BW500">>},{<<"codr">>,<<"4/5">>},{<<"lsnr">>,10.0},{<<"rssi">>,-52},{<<"size">>,16},{<<"data">>,<<"QAAAAEiAGgABQnoKS4pRkg==">>}]]}] from 12273815315514654977 on 59565
2020-08-04 18:33:37.198 [notice] <0.1321.0>@miner_lora:handle_packets:524 Routing {devaddr,1207959552}
2020-08-04 18:33:37.199 [info] <0.1316.0>@blockchain_state_channels_client:handle_packet:297 handle_packet {packet_pb,0,lorawan,<<64,0,0,0,72,128,26,0,1,66,122,10,75,138,81,146>>,676026522,-52,904.6,<<"SF8BW500">>,10.0,{routing_informatio
n_pb,{devaddr,1207959552}}} to [{routing_v1,1,<<1,125,102,101,5,171,157,180,249,144,239,214,128,120,131,69,171,50,195,32,115,252,39,208,164,32,152,211,182,230,216,194,136>>,[<<0,241,20,68,146,24,117,226,239,116,53,81,58,29,31,27,15,164,15
8,50,66,149,106,36,56,57,18,236,93,79,25,64,119>>],[<<193,92,2,137,236,45,10,145,10,0,0,0,0,0,0,0,0,0,112,34,183,127,0,0,0,0,0,0,0,0,0,0,72,236,132,14,0,112,0,0,0,0,0,0,0,0,0,0,1,0,0,0,48,101,0,0,0,0,0,0,0,0,0,0,49,0,0,0,54,0,0,0,58,0,0,0
>>],[<<0,0,0,127,255,0>>],0}]
2020-08-04 18:33:37.226 [error] <0.19758.9>@blockchain_state_channels_client:dial:518 failed to dial "/p2p/112qB3YaH5bZkCnKA5uRH7tBtGNv2Y5B4smv1jsmvGUzgKT71QpE":{exit,{normal,{gen_server,call,[<0.12516.9>,open]}}}
2020-08-04 18:33:37.227 [error] <0.1316.0>@blockchain_state_channels_client:handle_info:180 failed to dial 1: failed dropping 2 packets

It appears that the miner is trying to connect to a p2p address but the connection is failing. Instead of trying again with a different peer (as I would expect), the miner "gives up" and drops the packet.

Info

• Sensor: Adafruit Feather M0 RFM9x running code that is very similar to this example code.
• Packet forwarder: Raspberry Pi 3 B+ with a RAK Wireless Pilot Gateway Pro RAK7243
  • Set up using this tutorial
• Miner: AWS EC2 t2.small instance (Ubuntu 20.04 LTS x86)
  • Set up using this tutorial
  • The miner is running version: miner-amd64_2020.07.31.0_GA

Release validator node for testnet

Release testnet validator node for users to start testing.

No reply to join request

Hi, I hope it's the right channel. I set up my miner and it works. I also forward LoRaWAN traffic to udp:1680, it's well received but the server doesn't reply with join accept

2020-09-24 23:35:06.994 1 [info] <0.1256.0>@miner_lora:handle_udp_packet:380 PUSH_DATA [{<<"rxpk">>,[[{<<"aesk">>,0},{<<"brd">>,8},{<<"codr">>,<<"4/5">>},{<<"data">>,<<"AAAAAAAAAAAA8RAAAACyGACb+MNEmz4=">>},{<<"datr">>,<<"SF8BW500">>},{<<"freq">>,903.0},{<<"jver">>,2},{<<"lsnr">>,9.8},{<<"modu">>,<<"LORA">>},{<<"rsig">>,[[{<<"ant">>,0},{<<"chan">>,8},{<<"lsnr">>,9.8},{<<"rssic">>,-49}]]},{<<"rssi">>,-49},{<<"size">>,23},{<<"stat">>,1},{<<"tmms">>,1285025724978},{<<"tmst">>,291236404}]]}] from 8103944956391260201 on 45160
2020-09-24 23:35:06.994 1 [notice] <0.1256.0>@miner_lora:handle_packets:526 Routing {eui,6951112510804209,0}
2020-09-24 23:35:06.997 1 [info] <0.1250.0>@blockchain_state_channels_client:handle_packet:353 handle_packet {packet_pb,0,lorawan,<<0,0,0,0,0,0,0,0,0,241,16,0,0,0,178,24,0,155,248,195,68,155,62>>,291236404,-49,903.0,<<"SF8BW500">>,9.8,{routing_information_pb,{eui,{eui_pb,6951112510804209,0}}}} to ["/p2p/112qB3YaH5bZkCnKA5uRH7tBtGNv2Y5B4smv1jsmvGUzgKT71QpE","/p2p/1124CJ9yJaHq4D6ugyPCDnSBzQik61C1BqD9VMh1vsUmjwt16HNB"]

the AAAAA... packet is a join request and the join accept should be a I..... packet

miner keeps looping on pi4

docker run --env REGION_OVERRIDE=US915 --publish 1680:1680/udp --publish 44158:44158/tcp --name miner --mount type=bind,source=/home/apple/miner_data,target=/var/data quay.io/team-helium/miner:miner-arm64_2020.10.12.1_GA
standard_init_linux.go:207: exec user process caused "exec format error"

Broken Link: https://docs.helium.com/blockchain/run-your-own-miner returns a 404

Don't know if this is the right channel for this. If not, my apologies and please direct me accordingly.

The github readme.md for building a miner, in the paragraph 'miner step-by-step' has a line:

This is for development purposes only and if you are interested in running a miner, please follow our guide here.

With 'here' linking to https://docs.helium.com/blockchain/run-your-own-miner which returns a 404

Miner allows PUSH_DATA packets from multiple lora gateways

Currently, all hotspots on the Helium network are comprised of one miner and one lora gateway in a one-to-one mapping. However, the Lora server implementation in the Helium miner allows PUSH_DATA packets from multiple Lora gateways and will process them as PoC or data packets. I believe this could be potentially used to game PoC by having multiple gateways or multiple middlemen sending packets to a single miner.

PULL_RESPs (transmitting a packet) on the other hand are sent to a single gateway. The gateway to send packets to is selected by looking at the head of the gateway list which end up being the first gateway that sent a pull data request since the miner was up. I believe similar logic should be used to filter out PUSH_DATA packets sent by gateways.

Miner info --blocks-behind

I have no idea how difficult it would be to implement but it might be handy to show the delta between the miner's and the chain's respective "heights." Presuming such functionality is not already readily available, of course. Unfortunately, I lack the coding chops to tackle this and submit a PR.

Include version number of validator in peerbook

This will allow users via api/explorer to monitor their validator version number and stay updated.

Investigate hbbft txn buff loss after node restarts

The honeybadger transaction buffer is supposed to persist queued transactions, but it appears to not do so correctly. It would be good to understand why and to fix it if possible.

Create genesis block for testnet chain

Create the initial genesis block for the testnet chain.

make release fails with Error 1

Early in make release I see:

===> Compiling erlang_pbc
CMake Error at cmake/FindPBC.cmake:39 (target_include_directories):
Cannot specify include directories for imported target "PBC::PBC".
Call Stack (most recent call first):
CMakeLists.txt:104 (find_package)

CMake Error at cmake/FindPBC.cmake:47 (target_link_libraries):
Cannot specify link libraries for target "PBC::PBC" which is not built by
this project.
Call Stack (most recent call first):
CMakeLists.txt:104 (find_package)

-- Configuring incomplete, errors occurred!
See also "/root/miner/_build/default/lib/erlang_pbc/_build/c_src/CMakeFiles/CMakeOutput.log".
===> Hook for compile failed!

Makefile:32: recipe for target 'release' failed
make: *** [release] Error 1

What is Error 1?

Update miner to return swarm key as onboarding key

Update to return (and save) swarm key as onboarding key. This should make gateway-config pick it up as the onboarding key