Giter Site home page Giter Site logo

someguy123 / steem-docker Goto Github PK

View Code? Open in Web Editor NEW
41.0 16.0 32.0 272 KB

Steem-in-a-box - The easy way to manage Steem blockchain nodes (including witnesses, seeds and RPC nodes)

License: GNU Affero General Public License v3.0

Shell 94.29% Dockerfile 5.71%

steem-docker's Introduction

Steem-in-a-box by @someguy123

Steem-in-a-box is a toolkit for using the Steem docker images published by @someguy123.

It's purpose is to simplify the deployment of steemd nodes.

Features:

  • Automatic docker installer
  • Easily update Steem (steemd, cli_wallet etc.) with binary images
  • Easily build your own new versions of Steem by editing the docker files
  • Single command to download and install block_log from gtg's server
  • Easily adjust /dev/shm size
  • Automatically forwards port 2001 for seeds
  • Automatically installs a working example configuration for seeds, which can easily be customized for witnesses and full nodes
  • Quick access to common actions such as start, stop, replay, rebuild, local wallet, remote wallet, and much more
  • Constantly maintained by a top 20 Steem witness (@someguy123) - updated docker images are often available within 24 hours of version release

Usage

To install a witness or seed node:

git clone https://github.com/someguy123/steem-docker.git
cd steem-docker
# If you don't already have a docker installation, this will install it for you
./run.sh install_docker

# This downloads/updates the low-memory docker image for Steem
./run.sh install

# If you are a witness, you need to adjust the configuration as needed
# e.g. witness name, private key, logging config, turn off p2p-endpoint etc.
# If you're running a seed, then don't worry about the config, it will just work
nano data/witness_node_data_dir/config.ini

# (optional) Setting the .env file up (see the env settings section of this readme)
# will help you to adjust settings for steem-in-a-box
nano .env

# Once you've configured your server, it's recommended to download the block log, as replays can be
# faster than p2p download
./run.sh dlblocks

# You'll also want to set the shared memory size (use sudo if not logged in as root). 
# Adjust 64G to whatever size is needed for your type of server and make sure to leave growth room.
# Please be aware that the shared memory size changes constantly. Ask in a witness chatroom if you're unsure.
./run.sh shm_size 64G

# It's recommended to set vm.swappiness to 1, which tells the system to avoid using swap 
# unless absolutely necessary. To persist on reboot, place in /etc/sysctl.conf
sysctl -w vm.swappiness=1

# Then after you've downloaded the blockchain, you can start steemd in replay mode
./run.sh replay
# If you DON'T want to replay, use "start" instead
./run.sh start

You may want to persist the /dev/shm size (shared memory) across reboots. To do this, you can edit /etc/fstab, please be very careful, as any mistakes in this file will cause your system to become unbootable.

Simply add this to the bottom of the file on a new line. Be sure not to damage any other lines in the file. Adjust "64G" to whatever size you would like /dev/shm to be.

tmpfs   /dev/shm         tmpfs   nodev,nosuid,size=64G          0  0

Full node (RPC)

To install a full RPC node - follow the same steps as above, but use install_full instead of install.

Remember to adjust the config, you'll need a higher shared memory size (potentially up to 1 TB), and various plugins.

For handling requests to your full node in docker, I recommend spinning up an nginx container, and connecting nginx to the steem node using a docker network.

Example:

docker network create rpc_default
# Assuming your RPC container is called "rpc1" instead of witness/seed
docker network connect rpc_default rpc1
docker network connect rpc_default nginx

Nginx will now be able to access the container RPC1 via http://rpc1:8090 (assuming 8090 is the RPC port in your config). Then you can set up SSL and container port forwarding as needed for nginx.

Updating your Steem node

To update to a newer version of Steem, first check @someguy123's docker hub to see if a new version of Steem is uploaded. Low memory mode (witness/seed) images are tagged like "v0.20.0", while full node images are tagged as "v0.20.0-full".

Security updates may not be tagged under a specific version, instead latest/latest-full will simply show a newer "Last Updated" on docker hub.

If there is a new version available, then you can update using the following (be warned, a replay is needed in many cases):

git pull
./run.sh install
./run.sh restart

If you're updating a full node, please remember to use install_full instead of install.

If you experience issues during restart, try running replay instead. You may also want to check @someguy123's steemit for any special update instructions, such as config changes.

Checking the status of your node

You can use the logs command to see the output from steemd:

./run.sh logs

You can also connect the local wallet using:

./run.sh wallet

Be aware, you can't connect cli_wallet until your steemd has finished replaying.

Environment options

By default, run.sh will attempt to load variables from .env in the same directory as run.sh.

Anything which is not set in .env will fall back to a default value specified at the top of run.sh.

The most common .env which is recommended for witnesses is the following:

DOCKER_NAME=witness
PORTS=

The above .env file will set your docker container name to "witness", instead of the default "seed", and will also disable port forwarding, preventing exposure of the p2p port in the event you forget to turn off p2p-endpoint.

Full list of possible configuration options:

  • PORTS - default 2001 - a comma separated list of ports in steemd to forward to the internet
  • DOCKER_NAME - default seed - the container name to use for your steemd server
  • DOCKER_DIR - default $DIR/dkr - The directory to build the low memory node docker image from
  • FULL_DOCKER_DIR - default $DIR/dkr_fullnode - The directory to build the full-node RPC node docker image from
  • DK_TAG - default someguy123/steem:latest - The docker tag to obtain Steem from. Useful for installing beta versions, or downgrading to previous versions.
  • DK_TAG_FULL - default someguy123/steem:latest-full - The docker tag to obtain Steem (full RPC node) from. Useful for installing beta versions, or downgrading to previous versions.
  • SHM_DIR - default /dev/shm - override the location of shared_memory.bin and shared_memory.meta. /dev/shm is a RAM disk on Linux, and can be adjusted with shm_size
  • REMOTE_WS - default wss://steemd.privex.io - the websocket server to use for the remote_wallet command

Commands

Full list of ./run.sh commands:

  • start - start a stopped steem-docker instance
  • stop - shutdown a steem-docker instance
  • restart - restart a steem-docker instance (will also start it if it's already stopped)
  • wallet - connect to the local container wallet
  • remote_wallet - connect to a remote wallet (default is wss://steemd.privex.io - @privex load balancer)
    • be aware, due to the way load balancing works, this may raise an error at first. just run it again a few times and eventually it will connect. note that external servers often have a 60 second connection limit, so you may be disconnected within 60 seconds.
  • replay - replay a steem-docker instance (run stop first)
  • dlblocks - download blocks from gtg (gandalf)'s block log server and install them into the blockchain directory
  • shm_size (size) - change the size of /dev/shm, e.g. ./run.sh shm_size 64G for 64 gigabytes
  • install - install or update the steem docker image from docker hub
  • install_full - install or update the full node steem docker image from docker hub
  • build - build the low memory mode version of steem into a docker image from source
  • build_full - build the full node version of steem into a docker image from source
  • logs - display the logs of the container with automatic follow. press ctrl-c to exit
  • enter - open a bash prompt inside of the container for debugging

Steemit Post

Steemit Post for installing HF19: https://steemit.com/steem/@someguy123/your-guide-to-setting-up-a-witness-server-steem-in-a-box-hf19

LICENSE

Steem-in-a-box and the associated docker images were built by @someguy123 (github steemit twitter)

GNU Affero General Public License v3.0

SEE LICENSE FILE FOR MORE INFO

steem-docker's People

Contributors

azarus avatar lukestokes avatar mahdiyari avatar someguy123 avatar williambanks avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

steem-docker's Issues

./run.sh replay error

I have been good to version 0.19.6
but HF20(latest) version is some error.
the contents are as follows

./run.sh replay
Removing old container
seed
Running steem with replay...
8c2eba93bd44a7bc12d8d7c1bf3a72b4db9cf80bdbb89e8a8fcc78d559dd0a3c
Started.

docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8c2eba93bd44 steem "steemd --data-dir=/…" 9 seconds ago Exited (132) 8 seconds ago seed

The docker exited as soon as it starts
Can i found a solution?

Running into errors during the installation process

Following the instructions. Saw an error. Tried resizing an running the commands again. See bolded highlights below:

rolfedlugy-hegwer at RsMBP in ~/Documents
$ ssh -i "whopah.pem" [email protected]
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-1041-aws x86_64)

Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud

43 packages can be updated.
19 updates are security updates.

Last login: Fri Jan 5 18:43:02 2018 from num.num.num.num

ubuntu@ip-wha-wha-wha-wha:~$ cd steem-docker

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ ./run.sh install
Loading image from someguy123/steem
Using default tag: latest
latest: Pulling from someguy123/steem
d5c6f90da05d: Pull complete
1300883d87d5: Pull complete
c220aa3cfc1b: Pull complete
2e9398f099dc: Pull complete
dc27a084064f: Pull complete
f3e4adbb074a: Pull complete
5830076acf40: Pull complete
787ca1f89787: Pull complete
Digest: sha256:dlsafjhasdkjfldsakjflkjdsalkflasdkjflaskdjf
Status: Downloaded newer image for someguy123/steem:latest
Tagging as steem
Installation completed. You may now configure or run the server

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ ./run.sh dlblocks
Removing old block log
Download @gtg's block logs...
--2018-01-05 18:46:32-- https://gtg.steem.house/get/blockchain.xz/block_log.xz
Resolving gtg.steem.house (gtg.steem.house)... 51.254.67.207
Connecting to gtg.steem.house (gtg.steem.house)|51.254.67.207|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14762387884 (14G) [application/octet-stream]
Saving to: ‘/home/ubuntu/steem-docker/data/witness_node_data_dir/blockchain/block_log.xz’

/home/ubuntu/steem-docker/data/witness 38%[============================> ] 5.31G 8.32MB/s in 8m 52s

Cannot write to ‘/home/ubuntu/steem-docker/data/witness_node_data_dir/blockchain/block_log.xz’ (Success).
Decompressing block log... this may take a while...
xz: /home/ubuntu/steem-docker/data/witness_node_data_dir/blockchain/block_log: Write error: No space left on device
FINISHED. Blockchain downloaded and decompressed
Remember to resize your /dev/shm, and run with replay!
$ ./run.sh shm_size SIZE (e.g. 8G)
$ ./run.sh replay

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ ./run.sh remote_wallet
docker: Error response from daemon: write /var/lib/docker/containerd/daemon/io.containerd.metadata.v1.bolt/meta.db: no space left on device: unknown.

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ ./run.sh shm_size 16G
Setting SHM to 16G
mount: only root can use "--options" option

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ sudo ./run.sh shm_size 16G
Setting SHM to 16G

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ ./run.sh replay
Removing old container
Error: No such container: seed
Running steem with replay...
411f0440a45051969ea69d392636a42c82a2c327c2f2aa647a03c59cae67c20b
docker: Error response from daemon: write /var/lib/docker/containerd/daemon/io.containerd.metadata.v1.bolt/meta.db: no space left on device: unknown.
Started.

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ ./run.sh dlblocks
Removing old block log
Download @gtg's block logs...
--2018-01-05 19:03:42-- https://gtg.steem.house/get/blockchain.xz/block_log.xz
Resolving gtg.steem.house (gtg.steem.house)... 51.254.67.207
Connecting to gtg.steem.house (gtg.steem.house)|51.254.67.207|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14762387884 (14G) [application/octet-stream]
Saving to: ‘/home/ubuntu/steem-docker/data/witness_node_data_dir/blockchain/block_log.xz’

/home/ubuntu/steem-docker/data/witness 38%[============================> ] 5.31G 10.9MB/s in 6m 40s

Cannot write to ‘/home/ubuntu/steem-docker/data/witness_node_data_dir/blockchain/block_log.xz’ (Success).
Decompressing block log... this may take a while...
xz: /home/ubuntu/steem-docker/data/witness_node_data_dir/blockchain/block_log: Write error: No space left on device
FINISHED. Blockchain downloaded and decompressed
Remember to resize your /dev/shm, and run with replay!
$ ./run.sh shm_size SIZE (e.g. 8G)
$ ./run.sh replay

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ ./run.sh shm_size 16G
Setting SHM to 16G
mount: only root can use "--options" option

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ sudo ./run.sh shm_size 16G
Setting SHM to 16G

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$ sudo ./run.sh replay
Removing old container
seed
Running steem with replay...
74449414addc0ecd8e6591ceed584957b808eab2e2ce772809fd293a742b907b
docker: Error response from daemon: mkdir /var/lib/docker/containers/74449414addc0ecd8e6591ceed584957b808eab2e2ce772809fd293a742b907b/shm: no space left on device.
Started.

ubuntu@ip-wha-wha-wha-wha:~/steem-docker$

How to start a full node with mira enabled

Hi there,

I'm a bit confused how I would start a full node with mira enabled.
As far as I understand I need to change the docker tag in the .env file.

Which is the correct docker tag for that?
I see 4 possible tags, but none that seems quite right for me:

  • latest-mira, which is not a full node but has mira.
  • latest, which is not a full node and doesn't have mira.
  • lates-full, which is a full node but doesn't have mira.
  • mira-full, which is a full node and has mira, but hasn't been updated for 6 days and therefore doesn't have the latest patches/hardfork I suppose?

Thanks for clearing things up!

v0.20.0 replay stops

The replay stops at around 9%, and nothing more. I am kicked out of the docket logs at that point.

./run.sh replay
docker logs -f witness

1119343ms p2p_plugin.cpp:616            plugin_initialize    ] caught exception 0 exception: unspecified
process exited with: Host not found (authoritative)
    {"message":"Host not found (authoritative)"}
    asio.cpp:88 resolve_handler

    {}
    asio.cpp:168 resolve

    {"endpoint_string":"seed-central.steemit.com:2001"}
    p2p_plugin.cpp:73 resolve_string_to_ip_endpoints while adding seed node seed-central.steemit.com:2001
1119374ms p2p_plugin.cpp:616            plugin_initialize    ] caught exception 0 exception: unspecified
process exited with: Host not found (authoritative)
    {"message":"Host not found (authoritative)"}
    asio.cpp:88 resolve_handler

    {}
    asio.cpp:168 resolve

    {"endpoint_string":"seed-west.steemit.com:2001"}
    p2p_plugin.cpp:73 resolve_string_to_ip_endpoints while adding seed node seed-west.steemit.com:2001
1120286ms rc_plugin.cpp:919             plugin_initialize    ] Initializing resource credit plugin
1120286ms witness_plugin.cpp:648        plugin_initialize    ] Initializing witness plugin
1120286ms webserver_plugin.cpp:311      plugin_initialize    ] configured with 32 thread pool size
1120286ms webserver_plugin.cpp:343      plugin_initialize    ] configured http to listen on 0.0.0.0:8090
1120286ms webserver_plugin.cpp:349      plugin_initialize    ] configured ws to listen on 0.0.0.0:8090
------------------------------------------------------

            STARTING STEEM NETWORK

------------------------------------------------------
initminer public key: STM-----------------------
chain id: 0000000000000000000000000000000000000000000000000000000000000000
blockchain version: 0.20.0
------------------------------------------------------
1120287ms main.cpp:121                  main                 ] Backtrace on segfault is enabled.
1120287ms chain_plugin.cpp:405          plugin_startup       ] Starting chain with shared_file_size: 68719476736 bytes
1120287ms chain_plugin.cpp:483          plugin_startup       ] Replaying blockchain on user request.
1120287ms database.cpp:185              reindex              ] Reindexing Blockchain
1120317ms block_log.cpp:142             open                 ] Log is nonempty
1120317ms block_log.cpp:151             open                 ] Index is nonempty
1120317ms database.cpp:193              reindex              ] Replaying blocks...
1120317ms rc_plugin.cpp:465             operator()           ] Genesis pool_obj is {"id":0,"pool_array":["21640431401",165575358,"16030041350","14426954266915","5122556912307"]}
1120317ms rc_plugin.cpp:401             operator()           ] resource_new_accounts adjustment on block 1: -165574522
   0.385498%   100000 of 25940455   (65530M free)
   0.770996%   200000 of 25940455   (65530M free)
   1.15649%   300000 of 25940455   (65529M free)
   1.54199%   400000 of 25940455   (65528M free)
   1.92749%   500000 of 25940455   (65527M free)
   2.31299%   600000 of 25940455   (65526M free)
   2.69849%   700000 of 25940455   (65524M free)
   3.08399%   800000 of 25940455   (65522M free)
   3.46948%   900000 of 25940455   (65519M free)
1130187ms database.cpp:4708             apply_hardfork       ] HARDFORK 1 at block 905693
1131045ms database.cpp:4708             apply_hardfork       ] HARDFORK 2 at block 934585
1131589ms database.cpp:4708             apply_hardfork       ] HARDFORK 3 at block 953363
1132864ms legacy_asset.hpp:101          unpack               ] unpack legacy serialization 84959911236353
1132864ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1132864ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1132864ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1132864ms steem_evaluator.cpp:100       do_apply             ] Wrong fee symbol in block 994240
1132864ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1132864ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1132864ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
   3.85498%   1000000 of 25940455   (65516M free)
1133757ms legacy_asset.hpp:101          unpack               ] unpack legacy serialization 84959911236353
1133757ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1133757ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1133757ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1133757ms steem_evaluator.cpp:100       do_apply             ] Wrong fee symbol in block 1021529
1133757ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1133757ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1133757ms legacy_asset.hpp:70           pack                 ] pack legacy serialization 84959911236353
1134412ms database.cpp:4708             apply_hardfork       ] HARDFORK 4 at block 1041497
   4.24048%   1100000 of 25940455   (65514M free)
   4.62598%   1200000 of 25940455   (65513M free)
   5.01148%   1300000 of 25940455   (65512M free)
   5.39698%   1400000 of 25940455   (65510M free)
   5.78247%   1500000 of 25940455   (65508M free)
   6.16797%   1600000 of 25940455   (65506M free)
   6.55347%   1700000 of 25940455   (65504M free)
   6.93897%   1800000 of 25940455   (65500M free)
   7.32447%   1900000 of 25940455   (65497M free)
1165471ms database.cpp:4708             apply_hardfork       ] HARDFORK 5 at block 1934236
   7.70996%   2000000 of 25940455   (65493M free)
   8.09546%   2100000 of 25940455   (65490M free)
   8.48096%   2200000 of 25940455   (65487M free)
   8.86646%   2300000 of 25940455   (65483M free)
   9.25196%   2400000 of 25940455   (65479M free)
   9.63746%   2500000 of 25940455   (65475M free)

I was running on v.0.19.12 before this through compiling it myself.

Any idea what can be the reason for the abrupt stop?

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.