turboflakes/crunch

Crunch is a command-line interface (CLI) and Matrix Bot to claim staking rewards every Era for Substrate-based chains

crunch's Issues

Errors on paseo and kusama

Hi,

After updating to the latest version of crunch (0.15.0) kusama doesn't accept the rpc schema and throws an error. Strangely polkadot seems to be working with the same url schema:

May 22 11:03:27 worker05 crunch[810467]: [2024-05-22T09:03:27Z INFO crunch::crunch] Connected to Kusama network using wss://ksm-rpc.stakeworld.io:443 * Substrate node Parity Polkadot v1.12.0-b4016902ac7
May 22 11:04:57 worker05 crunch[810467]: [2024-05-22T09:04:57Z ERROR crunch::crunch] RPC error: relative URL without a base

kusama env file: CRUNCH_SUBSTRATE_WS_URL=wss://ksm-rpc.stakeworld.io:443
polkadot env: CRUNCH_SUBSTRATE_WS_URL=wss://dot-rpc.stakeworld.io:443

[2024-05-22T14:29:28Z INFO crunch::crunch] Connected to Polkadot network using wss://dot-rpc.stakeworld.io:443 * Substrate node Parity Polkadot v1.12.0-b4016902ac7
[2024-05-22T14:29:28Z INFO crunch::runtimes::polkadot] Inspect and crunch unclaimed payout rewards
[2024-05-22T14:29:29Z INFO crunch::runtimes::polkadot] 2 stashes loaded from 'config.stashes'

Also paseo seems to be crashing, I think already a little longer, related to activating async backing a while ago?

[2024-05-22T14:01:44Z INFO crunch::crunch] Connected to Paseo Testnet network using wss://pas-rpc.stakeworld.io:443 * Substrate node Parity Polkadot v1.12.0-b4016902ac7
thread 'async-std/runtime' panicked at src/crunch.rs:351:18:
internal error: entered unreachable code
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.7.0/src/task.rs:452:45:
task has failed

Subxt error: Metadata error: The generated code is not compatible with the node

When using the crunch tool v0.13.2 on the Polkadot-sdk v1.10.0 https://github.com/paritytech/polkadot-sdk/releases/tag/polkadot-v1.10.0 on the Kusama chain throws this error

[2024-04-20T01:15:07Z INFO  crunch] crunch v0.13.2 * Crunch is a command-line interface (CLI) to claim staking rewards (flakes) every X hours for Substrate-based chains

[2024-04-20T01:15:07Z INFO  crunch::crunch] Connected to Kusama network using ws://127.0.0.1:9941 * Substrate node Parity Polkadot v1.10.0-7049c3c9883

[2024-04-20T01:15:07Z INFO  crunch::runtimes::kusama] Inspect and `crunch` unclaimed payout rewards
[2024-04-20T01:15:07Z ERROR crunch::crunch] Subxt error: Metadata error: The generated code is not compatible with the node
thread 'async-std/runtime' panicked at src/crunch.rs:308:76:
called `Result::unwrap()` on an `Err` value: MatrixError("access_token not defined")

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Handle signals for clean exit

SIGINT and SIGTERM do not seem to cause crunch to exit cleanly. It would be nice to have a way to exit properly.

Example from running in podman

WARN[0010] StopSignal SIGTERM failed to stop container containercrunch in 10 seconds, resorting to SIGKILL 

Build error: unresolved import `async_task::Builder`

Seems like a missing dependency. Using rust 1.74

```unresolved import async_task::Builder
--> /home/payment/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.8.0/src/lib.rs:46:18
|
46 | use async_task::{Builder, Runnable};
| ^^^^^^^ no `Builder` in the root
|
= help: consider importing this struct instead:
std::thread::Builder

For more information about this error, try rustc --explain E0432.
error: could not compile async-executor (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

OpenSSL error, File not found

Starting from version 0.8.1 the following error is found
crunch: error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory.
It seems like support for OpenSSL v1 which is the default in Debian

Fetch stashes from remote endpoint

For the public channel it would be nice to have the option of <detail> <summary> available

Not working with my own matrix server.

It seem the bot doesn't work with self hosted matrix server... :(

Crunch crashes when there is already a transaction in the pool

Hi there! I want to report a small bug. When we initialize the Crunch Bot and it starts collecting rewards, if we kill and restart it, the bot crashes. This happens because the old transaction it made to collect rewards is still in the waiting pool. When we restart the bot, it tries to collect rewards again, but there is already a transaction (with the same fees) in the awaiting pool.

[2024-06-22T10:02:12Z ERROR crunch::crunch] Subxt error: Rpc error: RPC error: ErrorObject { code: ServerError(1014), message: "Priority is too low: (1 vs 1)", data: Some(RawValue("The transaction has too low priority to replace another transaction already in the pool.")) } thread 'async-std/runtime' panicked at src/crunch.rs:464:76:

Validator reward daily flag not respected

I'm trying to pay out KSM rewards daily for multiple stash addresses. Despite the daily flag, the payout was triggered every era. After increasing the batch size to 100, the payments don't happen after 24 hours. What am I misunderstanding?

Sep 21 11:28:28 crunch[1174245]: [2022-09-21T11:28:28Z INFO  crunch::report] 💯 Payout for era <del>4228</del> finalized at block #14547497
Sep 21 17:31:40 crunch[1312875]: [2022-09-21T17:31:40Z INFO  crunch::report] 💯 Payout for era <del>4229</del> finalized at block #14551089
Sep 21 23:32:11 crunch[1312875]: [2022-09-21T23:32:11Z INFO  crunch::report] 💯 Payout for era <del>4230</del> finalized at block #14554653

arguments:

ExecStart=/home/payment/crunch/target/release/crunch \
  --config-path /home/payment/stuffs/.env.kusama \
  --substrate-ws-url ws://127.0.0.1:9944 \
  rewards --short -f /home/payment/stuffs/.private_file \
 --disable-public-matrix-room \
 --disable-matrix daily

latest config

CRUNCH_SUBSTRATE_WS_URL=ws://127.0.0.1:9944
CRUNCH_MAXIMUM_PAYOUTS=84
CRUNCH_MAXIMUM_HISTORY_ERAS=84
CRUNCH_MAXIMUM_CALLS=100

Run the tool as systemd service

I try to run the tool as a systemd service, here is my try:

[Unit]
Description=Kusama Autopayout

[Service]
ExecStart=/home/tw/crunch-bot/crunch rewards --seed-path '/home/tw/crunch-bot/.private.seed' --disable-matrix
Restart=always
RestartSec=15

[Install]
WantedBy=multi-user.target

What I do wrong? How to pass also config file? Where I do mistake?

Support docker/podman

Add Dockerfile for basic docker/podman support

(If accepted, I'll happily update README.md as well)

Add ONE-T grades to crunch

Query ONE-T grades at the end of the era or reference that was not p/v

License conflict with subxt?

subxt is licensed as GPL v3. This project is MIT.

If one of the stashes have already been claimed could lead `crunch` to skip the next stashes payouts

Trying to use crunch with Polkadex substrate chain fails with Invalid Metadata error

Hi,

We are trying to use crunch (latest crunch version, built from source, tried the binary too) with Polkadex chain and it fails with following error:

$ crunch --config-path /home/user/.env.pdex --substrate-ws-url ws://127.0.0.1:9945/ view
[2022-02-11T12:31:56Z INFO crunch] crunch v0.4.1 * Crunch is a command-line interface (CLI) to claim staking rewards (flakes) every X hours for Substrate-based chains
[2022-02-11T12:31:56Z ERROR crunch::crunch] Invalid Metadata: Invalid version
[2022-02-11T12:31:56Z INFO crunch::crunch] Awaiting for connection using ws://127.0.0.1:9945/

---

We also tried to save polkadex metadata using subxt to metadata/polkadex_metadata.scale and compiled crunch again, but no go. Attaching polkadex_metadata.scale
polkadex_metadata.txt

We are currently using crunch successfully for kusama chain. Any help is appreciated.

Fails to run in Ubuntu 22.04 LTS

Hello @paulormart !!

Due to Ubuntu 22.04 LTS migrating from OpenSSL 1.1 to 3.0, when trying to run the current release in those systems, it returns:

crunch: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

Greetings!

'async-std/runtime' panicked

Hi all, i am getting this error when trying to start up crunch. Any idea how i can resolve it? Many thanks

[2023-05-10T06:39:16Z WARN crunch::runtimes::kusama] Crunch upgrade might be required soon. Local static metadata differs from current chain runtime version.
thread 'async-std/runtime' panicked at 'constructed from known-good static value; qed: InvalidFormat', src/crunch.rs:134:10
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
thread 'main' panicked at 'task has failed', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/async-task-4.3.0/src/task.rs:426:43

Add random wait time between batch calls

libssl.so.1.1 library issue on Ubuntu 22.04.2 LTS

The service fails to start with "/crunch-bot/crunch: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory...". Why am I getting this error? how do I fix this?

I am using [Crunch v0.9.3]

Cheers!

Pool permissionless claiming

Hi, I thought crunchbot would claim pool nominees rewards as well as compound them based on the new permissionless claiming feature, but apparently all it does is claim rewards for the validator the pool is nominating.

Are there any plans for such functionality?

Crunch suddenly doesn't work - Rpc error

I've been encountering this error lately:

INFO crunch::runtimes::kusama] Inspect and `crunch` unclaimed payout rewards
WARN crunch::runtimes::kusama] batch call indexes [0 : 2]
ERROR crunch::crunch] Subxt error: Rpc error: RPC error
INFOcrunch::crunch] Connected to Kusama network using ws://127.0.0.1:9944 * Substrate node Parity Polkadot v0.9.36-dc25abc712e
INFO crunch::matrix] The 'Crunch Bot' user @......t:matrix.org has been authenticated at matrix.org
INFO crunch::matrix] Messages will be sent to room #Y3J1bmNoL.............
INFOcrunch::runtimes::kusama] Inspect and `crunch` unclaimed payout rewards
WARN  crunch::runtimes::kusama] batch call indexes [0 : 2]
ERROR crunch::crunch] Subxt error: Rpc error: RPC error

Some discord member sugested to replace the RPC with
CRUNCH_SUBSTRATE_WS_URL=wss://ksm-rpc.stakeworld.io:443 which I did, but the error is still there and it is quite the same.

I was hoping that reinstalling the node will solve the issue so I reinstalled on a fresh Ubuntu. For some reason I decided to use 20.0.4 LTS, not 22.0.4 like I did before which seems to be a not very inspired choice: now I have another issue: the OpenSSL version is OpenSSL 1.1.1f which looks like it doesn't support crunch.
In the meantime I tried to upgrade my Openssl to 3.1.0, some errors occurred and I just gave up for now as it's Christmas Eve, lol.
I'll be back with a new report after OpenSSL will be upgraded to 3.1.0.

Subxt functions failing for Aleph Zero mainnet

Dear Paulo, I hope this case finds you well,

For your kind consideration, this is the issue we are facing with the mainnet of Aleph Zero, this network is implementing the runtime version aleph-node/46 and both, the local ws:// endpoint, and Crunch's metadata, are up to date with the appropriate versions.

The first symptom are shown in the logs as:

[WARN  crunch::runtimes::aleph_zero] Crunch upgrade might be required soon. Local static metadata differs from current chain runtime version.
[INFO  crunch::report] <details><summary>Next era expect <b>NO</b> rewards 🙁</summary>

(apart of the WARN, please note the absence of the Earlier claimed comment in the INFO block.

After which the bot runs without apparent problems and even hooks OK with the paidEras() event, but it fails to retrieve the pending payments and process no payouts at all.

Thus, with the help of another community member, it was determined that at least two functions seem to be failing in the aleph_zero.rs file:

• node_runtime::validate_codegen
• node_runtime::storage().staking().history_depth(), and/or
• crunch.client().clone().storage().fetch() for the above history_depth_addr

Do you have an idea were to start looking for a solution?

Please advise!

Best regards...

Milos

`Result::unwrap()` on an `Err` value: BadWordCount(1)

Since version v0.11.1 I get the following error:

Feb 28 23:33:43 goethe crunch[1340050]: thread 'async-std/runtime' panicked at src/crunch.rs:173:42:
Feb 28 23:33:43 goethe crunch[1340050]: called `Result::unwrap()` on an `Err` value: BadWordCount(1)
Feb 28 23:33:43 goethe crunch[1340050]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Feb 28 23:33:43 goethe systemd[1]: crunch-bot.service: Main process exited, code=exited, status=101/n/a
Feb 28 23:33:43 goethe systemd[1]: crunch-bot.service: Failed with result 'exit-code'.

The expected behavior is that crunchbot pays out my nominators, which doesn't happen because of the error.

The error happens on an amd64 Ubuntu 22.04 cloud server.
I can confirm the error happening on v0.11.1, v0.11.2 and v12.0. Currently I run v10.1 which doesn't have this problem.

Subxt error: Metadata

Hi All,
Firstly thanks for building this amazing bot.

I am facing this issue on Kusama after runtime upgrade to 0.9.42. Any idea how I can retrieve and change the metadata to get the bot working again? :(

[2023-05-25T07:01:49Z WARN crunch::runtimes::kusama] Crunch upgrade might be required soon. Local static metadata differs from current chain runtime version.
[2023-05-25T07:01:50Z ERROR crunch::crunch] Subxt error: Metadata: Pallet System Storage System has incompatible metadata

Authentification for stashes from remote url (Feature request)

in this function for remote url

pub async fn try_fetch_stashes_from_remote_url(
) -> Result<Option<Vec<String>>, CrunchError> {
    let config = CONFIG.clone();
    if config.stashes_url.len() == 0 {
        return Ok(None);
    }
    let response = reqwest::get(&config.stashes_url).await?.text().await?;
    let v: Vec<String> = response.split('\n').map(|s| s.to_string()).collect();
    if v.is_empty() {
        return Ok(None);
    }
    info!("{} stashes loaded from {}", v.len(), config.stashes_url);
    Ok(Some(v))
}

I would like to use some kind of authorization with this feature

e.g.

    [...]
    let client = Client::new();

    let user: Option<String> = &config.stashes_url_user;
    let password: Option<String> = &config.stashes_url_password;

    let response = client
        .get(&config.stashes_url)
        .basic_auth(user, password)
        .send();
   [...]