stefa2k/prysm-docker-compose

Run a stand-alone Prysm staking node with monitoring tools and own Ethereum 1 node out of the box!

prysm-docker-compose's Issues

Expose slasher's healthz in ha config

stefa2k/ansible-prysm#5
blocked by prysmaticlabs/prysm#6433

Support Medalla launchpad

https://docs.prylabs.network/docs/testnet/medalla

Upgrade to Grafana 7

Update release tag for alpha.12

Hi, looks like the latest stable release tag is alpha.12.

The git commit hash is 802bdf5, and the corresponding Docker tag is HEAD-802bdf.

I've put this in my .env file and it seems to work fine.

The validator still reports being alpha.11, but the corresponding commit hash seems to be correct:

$ docker-compose exec validator /app/validator/image.binary --version
validator version Prysm/v1.0.0-alpha.11/802bdf5561a8c713fc2a8ae5b16cdb02b192e94a. Built at: 2020-06-17 18:50:29+00:00

vs.

$ prysm.sh validator --version
Starting Prysm validator --version
validator version Prysm/v1.0.0-alpha.12/802bdf5561a8c713fc2a8ae5b16cdb02b192e94a. Built at: 2020-06-17 18:58:22+00:00

default .env file

Link dashboard repo instead of copy files

Use TLS for beacon, validator, slasher

Slashings because of timeout of slasher check (HA setup only)

Accounts were slashed due to haproxy check on node C for slasher of node A timed out (possibly due to this i think healthz can take 1000ms on certain scenarios (we have lock for span update) if those spans take long to update it might hold the lock statement by https://github.com/shayzluf)
Logs:
2277 slashing.txt

slashed validators: 2277, 2292, 2330, 2366, 2664, 2807, 2377, 2385, 2459, 2532, 2581, 2953, 2971, 3033

Integrate ELK

Use

• elasticsearch
• kibana
• logstash
• minimal setup

Write docs for

• minimal setup
• filebeat

Slasher protection

prysmaticlabs/prysm#6871

Restrict internet access of validator service

ARM64 support

Deal with grpc retries

Related to prysmaticlabs/prysm#6669

Using own ethereum 1 node

Use own geth node to connect to with the prysm beacon node.

Specify protocol for beacon ports

GuillaumeMiralles' dashboard outdated

ERROR: Version in "./docker-compose.yml" is unsupported.

I'm getting this error when trying to run any of the YAML files with docker-compose up:

ERROR: Version in "./docker-compose.yaml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

I'm running Ubuntu 18.04 LTS with the following Docker versions:

$ docker --version
Docker version 19.03.8, build afacb8b7f0
$ docker-compose --version
docker-compose version 1.17.1, build unknown

I've consulted https://docs.docker.com/compose/compose-file/, which tells me my engine should support up to version 3.8 of the YAML file format, but for some reason it doesn't.

I've also tried to replace the version number in the file(s) with some random lower versions, but then I get errors about certain extensions (such as logging) not being supported.

CI - branch strategy - prysm repo trigger

Concepts for:

• Continuous Integration
• Git branching strategy
• Versioning of PDC
• Prysm docker tag trigger (new docker tag triggers PDC)

add monitoring-host to beacon-no-geth.yaml

adding this line to /config/beacon-no-geth.yaml

monitoring-host: 0.0.0.0

Spadina support

Expose beacon's healthz in HA config

stefa2k/ansible-prysm#5

HA: Prevent double attestations/proposal by validator switching beacn node

Disable fast sync for mainnet

disable-initial-sync-verify-all-signatures in beacon.yamls should not be set to yes when run on mainnet.

Include currency converter for dashboards

https://github.com/GuillaumeMiralles/prysm-grafana-dashboard/blob/master/currency_converter.md

Add Config files for Zinken

Log inconsistency

prysmaticlabs/prysm#6595

Fix: prysmaticlabs/prysm#6600

High Availability

• Reverse proxy for backup services (geth + beacon)

• Beacon connects to reverse proxy

• Validator connects to reverse proxy

• Slasher connects to reverse proxy

• static IPs for containers OR no dns cache for haproxy

• Testing

• Create docker-compose.overide.yaml

• Documentation (will happen here stefa2k/ansible-prysm#5)

• Merge into master

createAccounts.sh not working for Onyx testnet (alpha.10/alpha.11)

Hi, first of all, thanks for the great work on this setup! I've spun up two Prysm validator setups for Onyx, one with the prysm.sh script, and one with this setup. The former was easier, but this one seems more flexible.

However, the createAccounts.sh script wasn't working for me (I know it comes without warranty!), so I took some time to figure out why. This is what I found:

1. First of all, the deposit contract address ($BEACON_CONTRACT in createAccounts.sh) is wrong. I believe this is the address for the now-defunct Topaz network). It should be 0x0F0F0fc0530007361933EaB5DB97d09aCDD6C1c8 for Onyx.
2. Even after changing the address, all the deposit transactions fail because they don't include the depositdata. This is because it's not properly extracted from the tempfile. Could be because of changes in Prysm, but the data is now located on line 8 instead of line 9. Changing createAccounts.sh:13 to RAWTXDATA=$(sed -n 8,1p /tmp/validatorcreateoutput | tr -d '\r') fixes this.
3. Even with the deposit address and depositdata correct, some transactions (about 5% or so) fail due to running out of gas. Members of the Prysm Discord suggested setting the gas limit to 400,000 to avoid this.

I've forked the repo and will send you a pull request with the suggest changes.

Keep local configuration separate from upstream

Summary

As mentioned in #20 (comment), if one uses this repo to manage a Prysm installation, it is somewhat cumbersome to keep up to date with changes to the repo while also maintaining local configuration changes, some of which (like passwords) are mandatory.

For instance, I should pick my own validator key password and set it in .env and validator/keystore.json, but if one of these files is updated in the repository, my local changes will prevent me from pulling these updates.

We should devise a method to keep local changes to any configuration files separate from the repository, similar to how a local docker-compose.override.yaml can be used to make changes to the docker-compose configuration without having to edit docker-compose.yml.

Details

So far, I have identified the following files to be likely to contain local changes that should survive an upstream update:

Definitely have local changes (password)

• .env (likely to change upstream)
• validator/keystore.json (unlikely to change upstream)

Possibly have local changes (external IP, graffiti)

• config/*.yaml

Decrease log size

Logs are now capped at 1gb per container (10x100mb)

• figure out a good value
• change settings

Insane load when using slasher in HA setup

Relates to prysmaticlabs/prysm#6519

Trying:

• enable-local-protection: true -> didn't help
• haproxy reties 0 on all backends -> didn't help
• validator --grpc-retries 0 -> didn't work
• validator enable-external-slasher-protection: false (was true) -> works
• haproxy connection limit to beacon backend
• slasher using --enable-historical-detection
• more ram -> works see prysmaticlabs/prysm#6519 (comment)

Medalla ready

https://github.com/goerli/medalla/tree/master/medalla

Add metanull's Grafana dashboards

https://github.com/metanull-operator/eth2-grafana

slasher.log gets too big

At the moment slasher.log is growing to 15gb in 24h.

Make all services by default active

Add grafana as a service

Make use of https://github.com/GuillaumeMiralles/prysm-grafana-dashboard/tree/master

Infura configuration for prysm-no-geth

I wonder how to configure an infura endpoint when launching prysm with no geth configuration. Does prysm point by default to https://goerli.prylabs.net?
Should you add http-web3provider in beacon-no-geth.yaml?

Adjust to accounts-v2

prysmaticlabs/prysm#6220

Enable db backup webhook

grafana errors - Data source not found

I see the grafana alerting constantly due to these errors:

grafana       | t=2020-08-24T18:55:24+0000 lvl=eror msg="Alert Rule Result Error" logger=alerting.evalContext ruleId=5 name="NODE: 50 errors  or 100 warns in 1h" error="Could not find datasource Data source not found" changing state to=alerting
grafana       | t=2020-08-24T18:55:24+0000 lvl=eror msg="Alert Rule Result Error" logger=alerting.evalContext ruleId=7 name="NETWORK: Participation rate below 66%" error="Could not find datasource Data source not found" changing state to=alerting
grafana       | t=2020-08-24T18:55:25+0000 lvl=eror msg="Alert Rule Result Error" logger=alerting.evalContext ruleId=4 name="VALIDATOR: Hourly earning <= 0" error="Could not find datasource

but the data is valid when inspecting and testing the alerts look ok.

Grafana settings doesn't survive restart

Adding a volume should solve this.