poanetwork/deployment-azure

Azure Templates for deploying POA Network to on Azure Cloud

deployment-azure's Issues

Add logrotate to the template

Add logrotate to prevent file system from out of space problems

Rights and folders

By default azure downloads and runs installation script in /var/lib/waagent/custom-script/download/0/ under root.
By default root doesn't have a password, i.e. you can run sudo any-command without password.

1. Should we move all necessary files to /home/$ADMIN_USERNAME and change files ownership to $ADMIN_USERNAME?
2. Should we make a password for sudo and ask it in cloud UI?
3. Should we run docker as non-root that is still equivalent to root?

Problem: Cannot locate spec.json

Info:
https://github.com/oraclesorg/test-templates/blob/AlphaTestTestNet/TestTestNet/bootnode/node.toml#L3

Where are you getting this spec from?

When entering deployment template info, address is provided twice

In the deployment template, we specify the mining address, which we also include as part of the mining keyfile JSON. That's a redundant data entry that the system can extract from the JSON, minimizing deployment friction.

Best, Michael

Patent rights

Hey guys,

It seems that your idea of Proof of authority conflicts with my pending patent application on Royal Network.

We both concluded that small networks would perform better than peer to peer. I saw that this idea hadn't been officialized, standardized or anyhow formalized, and so I filed for it, among other senses of my Confidence Coin blockchain.

While I choose to implement the selection of the peers by their mining power, in Royal Network only those who mined a block can join the network, I did offer several alternatives to the selection process in my application.

As I said it is just in a pending stage and it's not a patent yet, but it will be.

So FYI guys.

Best,
Ilya Gazman
confidence-coin.com

Add Ethereum Network Intelligence API

In installer of Parity (curl https://get.parity.io -Lk) there is a block about installing netstats

		[ ! -d "www" ] && git clone https://github.com/cubedro/eth-net-intelligence-api netstats
		oldpwd= $(pwd)
		cd netstats
		sudo npm install
		sudo npm install pm2 -g

		cat > app.json << EOL

Let's install the main node on the bootnode for the testnet and connect from each mining node. A secret will be one more manual field when a miner runs the template.

(Fix) Rename repository

Problem:
Repository has ambiguous name test and templates

• it contains not only test branches
• templates, it's more like scripts

Solution:

• rename repository to deployment-azure
  • deployment means this repo is used to deploy stuff
  • azure means this repo is used to deploy to Azure
• create an empty repository with test-template with README with a link to a new repository
• update wiki and instructions

Azure Standard_A1 server

In your config Standard A1 servers are listed. Do you use them when deploying a nodes?

1 core with 2.2Ghz at the nodes

The load increases with the number of blocks, then the parity was updated and the generation of blocks started from scratch. Within a few days the load on the processor again increased and the CPU did not suffice to generate new blocks.

As a result, the network goes into desynchronous and the nodes "fall off"

(Feature) Make SSH credentials optional for Validator's node

If no SSH credentials provided, then a validator's node started without ssh access to it.

Rationale:
for some validators, it's hard to keep the security of their ssh keys, especially on windows platform. For these validators having an option to run validator's node without ssh access will be a preferable option.

Why `--ui-no-validation`

For security reasons, I don't see why mining node should have this flag.
https://github.com/oraclesorg/test-templates/blob/AlphaTestTestNet/TestTestNet/mining-node/install.sh#L276

Everybody who wants to interact with OraclesPoA, should setup their own eth node.

Peers discovering

Something wrong with testnet http://104.46.34.192:3000/
Any suggestions?

sudo docker logs oracles-poa:

Loading config file from node.toml
2017-07-25 19:29:07 UTC main INFO parity::run  Starting Parity/v1.6.8-beta-c396229-20170608/x86_64-linux-gnu/rustc1.18.0
2017-07-25 19:29:07 UTC main INFO parity::run  State DB configuration: archive +Fat +Trace
2017-07-25 19:29:07 UTC main INFO parity::run  Operating mode: active
2017-07-25 19:29:07 UTC main INFO ethcore::service  Configured for SoundchainPoA using AuthorityRound engine
2017-07-25 19:29:07 UTC hyper-client INFO miner  Updated conversion rate to Ξ1 = US$198.72 (599072100 wei/gas)
2017-07-25 19:29:08 UTC main TRACE engine  Setting up contract caller.
2017-07-25 19:29:08 UTC main WARN parity::signer  *** INSECURE *** Running Trusted Signer with no origin validation.
2017-07-25 19:29:08 UTC main INFO parity::signer  If you do not intend this, exit now.
2017-07-25 19:29:09 UTC IO Worker #1 INFO network  Public node URL: enode://174faaad8f3eda7bd44638fc94e24c11055bc7ac60df3d711073c3fbe5671542983e08c9003460ff23c497a2fb7ca21ed9b0564021681660a46e5ef5099ce204@104.46.34.192:30300
2017-07-25 19:29:10 UTC IO Worker #3 DEBUG engine  Set of validators obtained: [f982f6ac73a26e243b7d26e0388b104817f75933, d5623777f13f9d16063b0dff850b026cb5da28fb]
2017-07-25 19:29:10 UTC IO Worker #3 TRACE engine  generate_seal: Issuing a block for step 300202190.
2017-07-25 19:29:15 UTC IO Worker #2 DEBUG engine  Set of validators obtained: [f982f6ac73a26e243b7d26e0388b104817f75933, d5623777f13f9d16063b0dff850b026cb5da28fb]
2017-07-25 19:29:15 UTC IO Worker #2 TRACE engine  generate_seal: Not a proposer for step 300202191.
2017-07-25 19:29:15 UTC Verifier #0 INFO import  Imported #4903 9867…56c5 (0 txs, 0.00 Mgas, 0.53 ms, 0.55 KiB) + another 1 block(s) containing 0 tx(s)
2017-07-25 19:29:15 UTC Verifier #0 DEBUG engine  Set of validators obtained: [f982f6ac73a26e243b7d26e0388b104817f75933, d5623777f13f9d16063b0dff850b026cb5da28fb]
2017-07-25 19:29:15 UTC Verifier #0 INFO reorg  Reorg to #4904 6749…7eac (3be8…29c9 #4902 417a…9d24 9867…56c5)
2017-07-25 19:29:15 UTC Verifier #0 DEBUG engine  Set of validators obtained: [f982f6ac73a26e243b7d26e0388b104817f75933, d5623777f13f9d16063b0dff850b026cb5da28fb]
2017-07-25 19:29:15 UTC Verifier #0 TRACE engine  generate_seal: Not a proposer for step 300202191.
2017-07-25 19:29:20 UTC IO Worker #2 TRACE engine  generate_seal: Issuing a block for step 300202192.
2017-07-25 19:29:20 UTC IO Worker #2 INFO import  Imported #4905 8ede…a3eb (0 txs, 0.00 Mgas, 1.73 ms, 1.10 KiB) + another 1 block(s) containing 0 tx(s)
2017-07-25 19:29:25 UTC IO Worker #1 DEBUG engine  Set of validators obtained: [f982f6ac73a26e243b7d26e0388b104817f75933, d5623777f13f9d16063b0dff850b026cb5da28fb]
2017-07-25 19:29:25 UTC IO Worker #1 TRACE engine  generate_seal: Not a proposer for step 300202193.

pm2 logs:

[TAILING] Tailing last 15 lines for [all] processes (change the value with --lines option)
/home/musereum-testnet/.pm2/pm2.log last 15 lines:
PM2        | 2017-07-25 12:24:02: Process dump file    : /home/musereum-testnet/.pm2/dump.pm2
PM2        | 2017-07-25 12:24:02: Concurrent actions   : 2
PM2        | 2017-07-25 12:24:02: SIGTERM timeout      : 1600
PM2        | 2017-07-25 12:24:02: ===============================================================================
PM2        | 2017-07-25 12:24:02: Starting execution sequence in -fork mode- for app name:netstats-daemon id:0
PM2        | 2017-07-25 12:24:02: App name:netstats-daemon id:0 online
PM2        | 2017-07-25 12:24:13: App [netstats-daemon] with id [0] and pid [15620], exited with code [1] via signal [SIGINT]
PM2        | 2017-07-25 12:24:13: Starting execution sequence in -fork mode- for app name:netstats-daemon id:0
PM2        | 2017-07-25 12:24:13: App name:netstats-daemon id:0 online
PM2        | 2017-07-25 16:50:48: App [netstats-daemon] with id [0] and pid [15746], exited with code [1] via signal [SIGINT]
PM2        | 2017-07-25 16:50:48: Starting execution sequence in -fork mode- for app name:netstats-daemon id:0

PM2        | 2017-07-25 16:50:48: App name:netstats-daemon id:0 online
PM2        | 2017-07-25 19:28:55: App [netstats-daemon] with id [0] and pid [41759], exited with code [1] via signal [SIGINT]
PM2        | 2017-07-25 19:28:55: Starting execution sequence in -fork mode- for app name:netstats-daemon id:0
PM2        | 2017-07-25 19:28:55: App name:netstats-daemon id:0 online

/home/musereum-testnet/logs/dashboard.err last 15 lines:
0|netstats | 2017-07-25 19:29:15 +00:00: [eth] ==> Got same block: 4903

/home/soundchain-testnet/logs/dashboard.out last 15 lines:
0|netstats | 2017-07-25 19:31:25.233 [API] [HIS] Got history from: alexeyElizarov
0|netstats | 2017-07-25 19:31:25.805 [API] [BLK] Block: 4930 from: soundchainPoATestnet
0|netstats | 2017-07-25 19:31:26.056 [API] [BLK] Block: 4930 from: alexeyElizarov
0|netstats | 2017-07-25 19:31:26.714 [API] [HIS] Got history from: soundchainPoATestnet
0|netstats | 2017-07-25 19:31:28.232 [API] [HIS] Got history from: alexeyElizarov
0|netstats | 2017-07-25 19:31:29.717 [API] [HIS] Got history from: soundchainPoATestnet
0|netstats | 2017-07-25 19:31:30.057 [API] [BLK] Block: 4931 from: alexeyElizarov
0|netstats | 2017-07-25 19:31:30.290 [API] [BLK] Block: 4931 from: soundchainPoATestnet
0|netstats | 2017-07-25 19:31:31.232 [API] [HIS] Got history from: alexeyElizarov
0|netstats | 2017-07-25 19:31:32.718 [API] [HIS] Got history from: soundchainPoATestnet
0|netstats | 2017-07-25 19:31:34.233 [API] [HIS] Got history from: alexeyElizarov
0|netstats | 2017-07-25 19:31:35.719 [API] [HIS] Got history from: soundchainPoATestnet
0|netstats | 2017-07-25 19:31:35.794 [API] [BLK] Block: 4932 from: soundchainPoATestnet
0|netstats | 2017-07-25 19:31:36.063 [API] [BLK] Block: 4932 from: alexeyElizarov
0|netstats | 2017-07-25 19:31:37.233 [API] [HIS] Got history from: alexeyElizarov

High consistent CPU usage

Seeing very high consistent CPU usage on my VM node:

Zombie transactions produced by Truffle

Parity version

[ubuntu@light oracles]$ parity -v
Parity
  version Parity/v1.7.0-beta-5f2cabd6e-20170727/x86_64-linux-gnu/rustc1.19.0

a command line to run parity

parity --config node.toml --chain spec.json

node.totml

[ubuntu@light oracles]$ cat node.toml 
[parity]
chain = "spec.json"
base_path = "parity"
[account]
unlock = ["0x00e7ae17eccc615bbe008633d39f8108bd06e10e"]
password = ["/home/ubuntu/c/oracles/pass.file"]
keys_iterations = 10240
[network]
bootnodes=["enode://886bc3f5f6f6258a89febdf31c71514e27b3647151e96e16bd4bdf6328884b3e8d41bcfb64095496e3701142c4b1f0e6047781d353674b61a45b023845ea742a@40.117.197.50:30300"]
port = 30300
discovery=true
allow_ips = "all"
[rpc]
cors = "all"
interface = "all"
hosts = ["all"]
port = 8545
apis = ["web3", "eth", "net", "personal", "parity", "parity_set", "traces", "rpc", "parity_accounts"]

after downloading spec.json and node.toml I created an account
sent 0.5 from faucet to this account
added account section to toml with a link to password
started parity without ui

[ubuntu@light tst]$ truffle init
Downloading project...
Project initialized.

  Documentation: http://truffleframework.com/docs

Commands:

  Compile: truffle compile
  Migrate: truffle migrate
  Test:    truffle test

[ubuntu@light tst]$ truffle migrate
Compiling ./contracts/ConvertLib.sol...
Compiling ./contracts/MetaCoin.sol...
Compiling ./contracts/Migrations.sol...
Writing artifacts to ./build/contracts

Using network 'development'.

Running migration: 1_initial_migration.js
  Deploying Migrations...
  ... 0x2bea9e0999d807af27976dd65c3a49e1e7e1d009dd0ee331911e7a50a660617a

created migration generates zombie txs

(Feature) Remove repo hardcoded owner from scripts

https://github.com/oraclesorg/deployment-azure/blob/dev-mainnet/nodes/netstats-server/install.sh#L22

https://github.com/oraclesorg/deployment-azure/blob/dev-mainnet/nodes/netstats-server/install.sh#L41

I'd like to have a variable, not hardcoded oraclesorg in the scripts so I can use my own fork

Also
Owner of insta-mined coins should not be hardcoded:

https://github.com/oraclesorg/deployment-azure/blob/dev-mainnet/nodes/netstats-server/install.sh#L122

(BUG) Cannot deploy netstat

Template validation error

The template deployment 'Microsoft.Template' is not valid according to the validation procedure. 
The tracking id is 'c3be9485-654e-4f6c-a006-a369173540de'. See inner errors for details. 
Please see https://aka.ms/arm-deploy for usage details. (Code: InvalidTemplateDeployment)

(Feature) Add nginx to boonode in Sokol configuration

Rationale:

• nginx allows to proxy RPC calls using https on 443 port
• proxying RPC using https allows connecting to MEW (myethereum wallet) and other services which require https

we can use nginx setup from kovan
https://github.com/kovan-testnet/rpc-node-setup

private key and certificate chain will be provided in a UI of azure while deploying the VM

Auto install security updates for docker host/docker guest

For Docker guest:
https://stackoverflow.com/a/26543374/1268964

The same can be used for the docker host

Deployment failed

Followed https://www.youtube.com/watch?v=LeYGKyMqSzM. Deployed from AlphaTestTestNet to Azure, into a new VM. Deployment succeeded, but my node isn't showing up under http://testnet.oracles.org:3000/.

Thanks, Michael

Payout script doesn't work

Doesn't work for Henry
mining
http://testnet.oracles.org:4000/account/0xc1a5b2e4b842b139f1f57601e0d38b764feb65b0
payout
0x290b4B2c0b60c37f829FD402529A997DBf4B027a

for Jeff:
payout
0x58c47f56905ad6c0ec519f57867be1117dbb3b9e

But works for Roman
payout
0x49c250F3f9D4e1fa2686AaF88427c83C8ef4297b

They used the same template but on a different time. One day for the cron job passed several times for both Henry and Jeff

Please

• review recent changes, what can cause that bug?
• provide a command for debugging on their host

Add additional entropy to docker host

e.g. https://www.digitalocean.com/community/tutorials/how-to-setup-additional-entropy-for-cloud-servers-using-haveged

(Feature) Use multiset in spec.json instead of safeContract

Problem:
Hard forking of validators contract requires modification of code if use structure like

        "validators": {
          "safeContract": "0xf472e0e43570b9afaab67089615080cf7c20018d"
        }

https://github.com/oraclesorg/oracles-scripts/blob/master/spec.json#L9

Solution:
use multiset feature
https://github.com/paritytech/parity/wiki/Validator-Set#multi-set

Concerns:

• need to test it with hardfork on Sokol

Enable time sync for host

E.g. https://www.digitalocean.com/community/tutorials/how-to-set-up-time-synchronization-on-ubuntu-16-04

(Refactor) Remove unused RPC from validator's node

Add swap to docker host

# add swap
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo swapon -s
sudo sh -c "printf '/swapfile   none    swap    sw    0   0\n' >> /etc/fstab"
sudo sh -c "printf 'vm.swappiness=10\n' >> /etc/sysctl.conf"
sudo sysctl vm.vfs_cache_pressure=50
sudo sh -c "printf 'vm.vfs_cache_pressure = 50\n' >> /etc/sysctl.conf"

(Feature) Move initial_keys repo as variable to common.vars

https://github.com/oraclesorg/deployment-azure/blob/dev-mainnet/nodes/owner/install.sh#L87

I'd like this line to be exported from variables.

(Feature) Restart parity client on process exit

Problem:
Parity client may crash because of many different reasons.
Solution:
Restart process of exit if it is not a restart of the service

Node that drops from the network consumed 100% CPU indefinitely

When the network went to new version, the old node went into 100% CPU frenzy, indefinitely. This can get expensive, especially, if hosted in the cloud. Node needs to implement an exponential back-off policy.

Best, Michael

Suggestion in template's UI/UX

Please review and implement suggestions by Jeff Flowers
https://docs.google.com/document/d/e/2PACX-1vTiDlkMZAwpSrXghD2ZqWV8NQqHqsW-vChOZmxTM1-mi3gUf2SCPQkg8pFeZkOFwQz9nMmT7INya3d0/pub

Add --force-sealing to parity client launch

The consensus can be ran with --force-sealing which ensures that blocks are produced even if there are no transactions. This is necessary for fault tolerance described.

https://github.com/paritytech/parity/wiki/Aura

AWS

Hmm can this run on AWS?

Specify key and password in cloud UI

Maybe there is a way to specify them in ENV variables?

Explicitly use 16.04 for docker image

(Fix) Set gas limit to 6700000 and gasPrice 1 Gwei

Problem:
Gas price is 4700000 and on mainnet and Kovan it's more.
Gas price on mainnet is 1gwei

Solution:
On Sokol and Trinity set gas limit 6700000 and 1gwei gas price.

Add autostart of parity and netstat after restart

Add autostart of parity and netstat after restart to improve user experience after restart

Testnet Codename Sokol

New settings:

• 25 validators + MoC (Master of Ceremony)
• bootnode is a separate server (connected to ethstats)
• stats server and explorer is a separate server (connected to ethstats)
• MoC node (connected to the ethstats)

Features:

• Parity 1.8
• more resources on bootnode host
• clear name of hosts for (stats, bootnode, MoC node)
• no tracing
• logrotate 12hr
• no logentries (our account will be closed soon and I think about another solutions)
• more complex netstats password

Let's use Sokol as a branch name