counterpartyxcp/federatednode

Federated Node Build System

federatednode's Issues

Move to Python 3.4.2?

I think we should start testing Python 3.4.2 and if it behaves well make that a system requirement.
Or maybe 3.4.3 (currently rc1) so that we don't have to revisit this again.

https://docs.python.org/3/whatsnew/changelog.html

Missing counterparty-cli dependencies

counterparty-cli doesn't work (see CounterpartyXCP/counterparty-core#573).
It seems while stuff was moved to counterparty-cli, the deps didn't. I had to install the following:

python-dateutils
prettytable
requests
pysha3
bitcoin

I got this far but then ripemd was missing and pycrypto is already installed, so...

C:\Windows\system32>python -V
Python 3.4.2
C:\Windows\system32>counterpartyd -V
C:\Windows\system32>echo off
counterpartyd v9.49.3

I should mention that I suspect that this may be due to the fact that I'm testing Python 3.4.2. I just installed the binary Pycrypto package (which I had installed for Python 3.4.1) and installed the same version of pycrypto (2.6.1) as before and the problem still occurs, so I'm curious if other are seeing the same on Windows with Python 3.4.1.

v9.46: No such file or directory: '/home/xcp/.local/share/counterpartyd'

I got this after "stand-alone" counterpartyd 9.46.0 installation.
counterpartyd is executed with counterpartyd server and subdirectory .local never used to appear.

xcp@raspberrypi:~$ counterpartyd -V
counterpartyd v9.46.0
xcp@raspberrypi:~$ counterpartyd server
Traceback (most recent call last):
  File "/home/xcp/counterpartyd_build/dist/counterpartyd/counterpartyd.py", line 741, in <module>
    force=args.force)
  File "/home/xcp/counterpartyd_build/dist/counterpartyd/counterpartyd.py", line 261,
 in set_options if not os.path.isdir(config.DATA_DIR): os.mkdir(config.DATA_DIR)
FileNotFoundError: [Errno 2] No such file or directory: '/home/xcp/.local/share/counterpartyd'

(The issue apparently also appears in setup_federated_node.py - CounterpartyXCP/counterparty-core#423)

Note: this is on Debian, but earlier versions of coutnerpartyd (up to 9.35?) used to work fine.

Lines should be wrapped for better usability of docs

From the "Wrap Your Lines" department:

It's really a Github issue rather than any kind of bug, but this is to let everyone know that we may need to start line-wrapping online docs otherwise their PDF versions (presumably, great for offline use) will remain much less useful and HTML pages will continue to require horizontal scrolling.

Source:
https://media.readthedocs.org/pdf/counterpartyd-build/latest/counterpartyd-build.pdf

NameError: name 'logger' is not defined

If neither Y or N is pressed on Start counterpartyd automatically on system startup? (y/n), this error happens.

Successfully installed appdirs prettytable python-dateutil flask json-rpc pytest pycoin requests Flask-HTTPAuth tornado pycrypto tendo python-bitcoinlib pytest-cov Werkzeug itsdangerous py certifi cov-core coverage
Cleaning up...
2014-11-18 09:31:01,316|DEBUG: RUNNING COMMAND: ln -sf /home/xcp/counterpartyd_build/run.py /usr/local/bin/counterpartyd
Start counterpartyd automatically on system startup? (y/n): m
Traceback (most recent call last):
  File "setup.py", line 475, in <module>
    main()
  File "setup.py", line 469, in main
    setup_startup(paths, run_as_user, with_counterblockd, with_testnet, noninteractive)
  File "setup.py", line 282, in setup_startup
    logger.error("Please enter 'y' or 'n'")
NameError: name 'logger' is not defined

Clean OSl: FedNode - It appears this setup has been run already

• Install, update and upgrade U14.04
• Install git

xcp@ubuntu:~$ BRANCH=master
xcp@ubuntu:~$ cd ~ && wget -q https://raw.github.com/CounterpartyXCP/counterpartyd_build/${BRANCH}/setup_federated_node.py https://raw.github.com/CounterpartyXCP/counterpartyd_build/${BRANCH}/setup_util.py && sudo python3 setup_federated_node.py
It appears this setup has been run already. (r)ebuild node, or just (U)pdate from git? (r/U):

federate script error - setup_federated_node.py

sudo python3 setup_federated_node.py
Traceback (most recent call last):
File "setup_federated_node.py", line 885, in
main()
File "setup_federated_node.py", line 832, in main
if answered_questions['op'] == 'u': #just refresh counterpartyd, counterblockd, and counterwallet, etc. from github
KeyError: 'op'

Federated Node Rebuild from dev branch fails after issue #78

Building from 'develop' branch after having failed to build from master (see issue #78).
I chose to rebuild, develop, test and main, Counterwallet, Bitcore, no email, no hardening..

Ign http://us.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 2,799 B in 18s (152 B/s)
Reading package lists... Done
2014-08-23 16:16:23,103|DEBUG: RUNNING COMMAND: apt-get -y install git-core software-properties-common python-software-properties build-essential ssl-cert ntp
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: The package armory needs to be reinstalled, but I can't find an archive for it.
2014-08-23 16:16:23,662|ERROR: Command failed: 'apt-get -y install git-core software-properties-common python-software-properties build-essential ssl-cert ntp'

Document uninstallation procedure for supported OS

A proper uninstallation procedure should be documented.
Users who fail to install or who want to uninstall the s/w should be able to find that information in official documentation.

Federated node update failure

been trying to update our federated node, but if fails with "cp: cannot stat ‘/home/xcp/counterpartyd_build/dist/linux/init/bitcoind.conf.template’: No such file or directory"

location of the setup script is
/home/counterparty/counterpartyd_build/setup_federated_node.py

following are the setup inputs
counterparty@:~/counterpartyd_build$ sudo ./setup_federated_node.py
Enter the number for the role you want to build:
1: Counterwallet server
2: Vending machine
3: Blockexplorer server
4: counterpartyd-only
5: BTCPay Escrow Server
Your choice: 4
Building a Counterpartyd server
Build from branch (M)aster or (d)evelop? (M/d): M
Working with branch: master
Run as (t)estnet node, (m)ainnet node, or (B)oth? (t/m/B): m
Setting up to run on mainnet
Blockchain services, use (B)lockr.io (remote) or (i)nsight (local)? (B/i): i
Using insight
Enable public Counterpartyd setup (listen on all hosts w/ rpc/1234 user) (Y/n): n
Set up security hardening? (Y/n): n

Install log, after package downloads
2014-10-26 15:58:31,525|DEBUG: RUNNING COMMAND: adduser counterparty xcp
The user counterparty' is already a member ofxcp'.
2014-10-26 15:58:31,578|INFO: Checking out/updating counterpartyd_build:master from git...
2014-10-26 15:58:31,579|DEBUG: RUNNING COMMAND: cd ~xcp/counterpartyd_build && git pull origin master
From https://github.com/CounterpartyXCP/counterpartyd_build

• branch master -> FETCH_HEAD
  Already up-to-date.
  2014-10-26 15:58:32,392|DEBUG: RUNNING COMMAND: cd ~xcp/counterpartyd_build && git config core.sharedRepository group && find ~xcp/counterpartyd_build -type d -print0 | xargs -0 chmod g+s
  2014-10-26 15:58:32,403|DEBUG: RUNNING COMMAND: chown -R xcp:xcp ~xcp/counterpartyd_build
  2014-10-26 15:58:32,410|DEBUG: RUNNING COMMAND: chmod -R u+rw,g+rw,o+r,o-w ~xcp/counterpartyd_build
  2014-10-26 15:58:32,415|DEBUG: RUNNING COMMAND: cp -af /home/xcp/counterpartyd_build/dist/linux/other/xcpd_security_limits.conf /etc/security/limits.d/
  2014-10-26 15:58:32,419|DEBUG: RUNNING COMMAND: rm -rf /tmp/bitcoind.tar.gz /tmp/bitcoin-0.9.2.1-linux
  2014-10-26 15:58:32,421|DEBUG: RUNNING COMMAND: wget -O /tmp/bitcoind.tar.gz https://bitcoin.org/bin/0.9.2.1/bitcoin-0.9.2.1-linux.tar.gz
  --2014-10-26 15:58:32-- https://bitcoin.org/bin/0.9.2.1/bitcoin-0.9.2.1-linux.tar.gz
  Resolving bitcoin.org (bitcoin.org)... 208.64.123.130
  Connecting to bitcoin.org (bitcoin.org)|208.64.123.130|:443... connected.
  HTTP request sent, awaiting response... 200 OK
  Length: 36108494 (34M) [application/octet-stream]
  Saving to: ‘/tmp/bitcoind.tar.gz’

100%[======================================================================================================================================>] 36,108,494 12.1MB/s in 2.9s

2014-10-26 15:58:36 (12.1 MB/s) - ‘/tmp/bitcoind.tar.gz’ saved [36108494/36108494]

2014-10-26 15:58:36,038|DEBUG: RUNNING COMMAND: tar -C /tmp -zxvf /tmp/bitcoind.tar.gz
bitcoin-0.9.2.1-linux/
bitcoin-0.9.2.1-linux/bin/
bitcoin-0.9.2.1-linux/src/
bitcoin-0.9.2.1-linux/src/bitcoin-0.9.2.tar.gz
bitcoin-0.9.2.1-linux/bin/32/
bitcoin-0.9.2.1-linux/bin/64/
bitcoin-0.9.2.1-linux/bin/64/bitcoin-cli
bitcoin-0.9.2.1-linux/bin/64/bitcoin-qt
bitcoin-0.9.2.1-linux/bin/64/bitcoind
bitcoin-0.9.2.1-linux/bin/64/test_bitcoin
bitcoin-0.9.2.1-linux/bin/64/test_bitcoin-qt
bitcoin-0.9.2.1-linux/bin/32/bitcoin-cli
bitcoin-0.9.2.1-linux/bin/32/bitcoin-qt
bitcoin-0.9.2.1-linux/bin/32/bitcoind
bitcoin-0.9.2.1-linux/bin/32/test_bitcoin
bitcoin-0.9.2.1-linux/bin/32/test_bitcoin-qt
2014-10-26 15:58:37,602|DEBUG: RUNNING COMMAND: cp -af /tmp/bitcoin-0.9.2.1-linux/bin/64/bitcoind /usr/bin
2014-10-26 15:58:37,648|DEBUG: RUNNING COMMAND: cp -af /tmp/bitcoin-0.9.2.1-linux/bin/64/bitcoin-cli /usr/bin
2014-10-26 15:58:37,674|DEBUG: RUNNING COMMAND: rm -rf /tmp/bitcoind.tar.gz /tmp/bitcoin-0.9.2.1-linux
2014-10-26 15:58:37,740|DEBUG: RUNNING COMMAND: mkdir -p ~xcp/.bitcoin ~xcp/.bitcoin-testnet
2014-10-26 15:58:37,770|DEBUG: RUNNING COMMAND: rm -f /etc/init/bitcoin.conf /etc/init/bitcoin-testnet.conf
2014-10-26 15:58:37,773|DEBUG: RUNNING COMMAND: cp -af /home/xcp/counterpartyd_build/dist/linux/init/bitcoind.conf.template /etc/init/bitcoind.conf
cp: cannot stat ‘/home/xcp/counterpartyd_build/dist/linux/init/bitcoind.conf.template’: No such file or directory
2014-10-26 15:58:37,776|ERROR: Command failed: 'cp -af /home/xcp/counterpartyd_build/dist/linux/init/bitcoind.conf.template /etc/init/bitcoind.conf'

counterwallet desktop

get a desktop counterwallet running:

• under windows and macos, user downloads boot2docker (https://github.com/boot2docker/boot2docker)
• user then grabs federatednode/counterwallet_simple image and boots that
• user can then hit the webpage and get the desktop experience!
• this all can be packaged into an automated installer/process probably, for windows and macosx, , which will install boot2docker, then download the counterwallet-simple image, and install shortcuts to launch counterwallet. when the shortcut is clicked, the boot2docker will start up the counterwallet image, and launch a web browser to the counterwallet site (served locally by the image)
• on linux, the user can just apt-get (or whatever) install docker, and grab the image that way

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

apsw.cpython-34m-x86_64-linux-gnu.so: undefined symbol: sqlite3_enable_load_extension

Python:

• Python 2.7.6
• Python 3.4.0

OS:

• Linux ubuntu 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

I install from master using setup.py and then get this error.

xcp@ubuntu:~/counterpartyd_build$ counterpartyd server
Traceback (most recent call last):
  File "/home/xcp/counterpartyd_build/dist/counterpartyd/counterpartyd.py", line 24, in <module>
    from lib import config, api, util, exceptions, bitcoin, blocks, blockchain
  File "/home/xcp/counterpartyd_build/dist/counterpartyd/lib/api.py", line 16, in <module>
    import apsw
ImportError: /usr/lib/python3/dist-packages/apsw.cpython-34m-x86_64-linux-gnu.so: undefined symbol: sqlite3_enable_load_extension

sv stop iwatch fail: iwatch: unable to change to service directory: file does not exist

When executing setup_federated_node.py for 9.45.0 on U14.04:

....
        Using blockr.io
Email address where support cases should go (blank to disable):
Set up security hardening? (Y/n): n
Autostart services (including on boot)? (y/N): N
2014-10-28 12:55:47,324|DEBUG: answered_questions: {'backend_rpc_mode': 'b', 'run_mode': 'b', 'op': 'r', 'autostart_services': 'n', 'security_hardening': 'n', 'counterpartyd_public': None, 'counterwallet_support_email': '', 'role': 'counterwallet', 'branch': 'master', 'blockchain_service': 'b'}
2014-10-28 12:55:47,324|WARNING: STOPPING SERVICES
2014-10-28 12:55:47,324|DEBUG: RUNNING COMMAND: sv stop iwatch
fail: iwatch: unable to change to service directory: file does not exist

Avoid package (bitcoind, armory) re-download

setup.py could check if the files that were downloaded before can be used. It can save time when rebuilding. This is a low priority item, of course.

2014-11-04 05:04:28,568|DEBUG: RUNNING COMMAND: rm -rf /tmp/bitcoind.tar.gz /tmp/bitcoin-0.9.2.1-linux
2014-11-04 05:04:28,570|DEBUG: RUNNING COMMAND: wget -O /tmp/bitcoind.tar.gz https://bitcoin.org/bin/0.9.2.1/bitcoin-0.9.2.1-linux.tar.gz

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Add setup option to pull XCP DB

A special setup option (setup.py clean?) would pull a recent DB (a new one could be posted on Github after every release) and save the user some time.
setup.py could perform a basic sanity check: if there's no existing configuration or DB, that means someone either deleted them (=clean install) or installed elsewhere (=power user).

No way to uninstall counterpartyd

My system is full with garbage due to issue #2 And I have no way to remove it.
There is no documentation anywhere

Bootstrapping databases

The bootstrapping databases don't need to be packaged with the *.db-shm or *.db-wal files, as long as they are closed cleanly. The tarballs should be much smaller, then.

Also, the bootstrapping files need to be made much easier to find. Right now they're only here?

counterpartyd v9.34.0 can't build binary on Windows because of missing exe.win-i386-3.2 directory

In this configuration (32-bit Python, 64-bit WIndows) python setup.py build doesn't work.

C:\counterpartyd_build>python -V
Python 3.3.5

C:\counterpartyd_build>counterpartyd -V

C:\counterpartyd_build>echo off
counterpartyd v9.34.0

This is the probem - it seems something about final building steps.

copying C:\Python33\lib\site-packages\win32\win32wnet.pyd -> C:\counterpartyd_build\bin\exe.win32-3.3\win32wnet.pyd
Traceback (most recent call last):
  File "C:\Python33\lib\shutil.py", line 523, in move
    os.rename(src, real_dst)
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\counterpartyd_build\\bin\\build'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 555, in <module>
    main()
  File "setup.py", line 536, in main
    do_build(paths, with_counterblockd)
  File "setup.py", line 438, in do_build
    shutil.move(os.path.join(paths['bin_path'], "exe.win-%s-%s" % (arch, PYTHON3_VER)), os.path.join(paths['bin_path'], "build"))
  File "C:\Python33\lib\shutil.py", line 535, in move
    copy2(src, real_dst)
  File "C:\Python33\lib\shutil.py", line 243, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Python33\lib\shutil.py", line 109, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\counterpartyd_build\\bin\\exe.win-i386-3.2'

I don't know what's 3.2 in exe.win-i386-3.2 supposed to mean, but my Python is v3.3.5.
Interestingly, exe.win32-3.3 is created in the same directory and I think this is because the program can't get the same architecture info from OS and Python. The OS is a 64-bit Windows 7, while Python is 3.3 but 32-bit.

counterpartyd 9.47 on Ubuntu 12.04 requires these additional steps

It requires these preparatory steps:

sudo apt-get install curl -y
wget http://mirror.pnl.gov/ubuntu//pool/main/l/leveldb/libleveldb1_1.17-1_amd64.deb
wget http://mirror.pnl.gov/ubuntu//pool/main/l/leveldb/libleveldb-dev_1.17-1_amd64.deb
sudo dpkg -i libleveldb1_1.17-1_amd64.deb
sudo dpkg -i libleveldb-dev_1.17-1_amd64.deb

Separately, there's something wrong with flask and that would need to be fixed in the code.

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.3/threading.py", line 901, in _bootstrap_inner
    self.run()
  File "/home/counterparty/counterpartyd_build/dist/counterpartyd/lib/api.py", line 317, in run
    app = flask.Flask(__name__)
  File "/home/counterparty/counterpartyd_build/env/lib/python3.3/site-packages/flask/app.py", line 331, in __init__
    instance_path = self.auto_find_instance_path()
  File "/home/counterparty/counterpartyd_build/env/lib/python3.3/site-packages/flask/app.py", line 622, in auto_find_instance_path
    prefix, package_path = find_package(self.import_name)
  File "/home/counterparty/counterpartyd_build/env/lib/python3.3/site-packages/flask/helpers.py", line 682, in find_package
    if loader.is_package(root_mod_name):
AttributeError: 'NamespaceLoader' object has no attribute 'is_package'

What's the problem? See https://github.com/mitsuhiko/flask/blob/6fcc2ae3f4536e4d2651c1a00abe2306212a5bfd/flask/helpers.py:

def _matching_loader_thinks_module_is_package(loader, mod_name):
    """Given the loader that loaded a module and the module this function
    attempts to figure out if the given module is actually a package.
    """
    # If the loader can tell us if something is a package, we can
    # directly ask the loader.
    if hasattr(loader, 'is_package'):
        return loader.is_package(mod_name)
    # importlib's namespace loaders do not have this functionality but
    # all the modules it loads are packages, so we can take advantage of
    # this information.
    elif (loader.__class__.__module__ == '_frozen_importlib' and
          loader.__class__.__name__ == 'NamespaceLoader'):
        return True
    # Otherwise we need to fail with an error that explains what went
    # wrong.
    raise AttributeError(
        ('%s.is_package() method is missing but is required by Flask of '
         'PEP 302 import hooks.  If you do not use import hooks and '
         'you encounter this error please file a bug against Flask.') %
        loader.__class__.__name__)

I fixed this in a roundabout way - by butchering setup.py and changing it to use python 3.4.
Before that I removed python3.3 packages and changed to python3.4:

sudo add-apt-repository ppa:fkrull/deadsnakes
sudo apt-get update
sudo apt-get install python3.4 python3.4-dev libpython3.4-dev

You may need to remove symbolic links from python to python3.3 and change them to point to python3.4.

sudo ln -s /usr/bin/python3.4 /usr/bin/python3

Add `counterparty-cli` to $PATH automatically

Confusing info on counterpartyd in Setting up Bitcoind section

In the section Setting up Bitcoind* we have a mention of counterparty configuration which seems out of context. Can that be clarified or perhaps removed from chapter on getting wtarted with Bitcoind?

• http://counterpartyd-build.readthedocs.org/en/latest/SettingUpBitcoind.html#on-windows

---

Once done, press CTRL-S to save, and close Notepad. The config file will be saved here::
%AppData%\Roaming\Counterparty\counterpartyd\counterpartyd.conf

counterpartyd v9.34.0 "setup.py build" runs into Python cherrypy-wsgiserver bug

    building 'markupsafe._speedups' extension
    C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\cl.exe /c /nologo
 /Ox /MD /W3 /GS- /DNDEBUG -IC:\Python33\include -IC:\Python33\include /Tcmarkup
safe/_speedups.c /Fobuild\temp.win32-3.3\Release\markupsafe/_speedups.obj
    _speedups.c
    C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /no
logo /INCREMENTAL:NO /LIBPATH:C:\Python33\Libs /LIBPATH:C:\counterpartyd_build\e
nv\libs /LIBPATH:C:\counterpartyd_build\env\PCbuild /EXPORT:PyInit__speedups bui
ld\temp.win32-3.3\Release\markupsafe/_speedups.obj /OUT:build\lib.win32-3.3\mark
upsafe\_speedups.pyd /IMPLIB:build\temp.win32-3.3\Release\markupsafe\_speedups.l
ib /MANIFESTFILE:build\temp.win32-3.3\Release\markupsafe\_speedups.pyd.manifest
       Creating library build\temp.win32-3.3\Release\markupsafe\_speedups.lib an
d object build\temp.win32-3.3\Release\markupsafe\_speedups.exp
Successfully installed prettytable python-dateutil flask json-rpc pytest pycoin
requests Flask-HTTPAuth tornado six Werkzeug Jinja2 itsdangerous py colorama cer
tifi markupsafe
Cleaning up...
2014-07-25 12:44:14,004|DEBUG: Cleaning any old build dirs...
2014-07-25 12:44:14,005|DEBUG: RUNNING COMMAND: C:\counterpartyd_build\env\Scrip
ts\python.exe "C:\counterpartyd_build\dist\_cxfreeze_setup.py" build -b "C:\coun
terpartyd_build\bin"
running build
running build_exe
Traceback (most recent call last):
  File "C:\counterpartyd_build\dist\_cxfreeze_setup.py", line 24, in <module>
    executables = [Executable(os.path.join(scriptPath, "counterpartyd.py"), base
=base)])
  File "C:\Python33\lib\site-packages\cx_Freeze\dist.py", line 362, in setup
    distutils.core.setup(**attrs)
  File "C:\Python33\Lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Python33\Lib\distutils\dist.py", line 930, in run_commands
    self.run_command(cmd)
  File "C:\Python33\Lib\distutils\dist.py", line 949, in run_command
    cmd_obj.run()
  File "C:\Python33\Lib\distutils\command\build.py", line 126, in run
    self.run_command(cmd_name)
  File "C:\Python33\Lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "C:\Python33\Lib\distutils\dist.py", line 949, in run_command
    cmd_obj.run()
  File "C:\Python33\lib\site-packages\cx_Freeze\dist.py", line 232, in run
    freezer.Freeze()
  File "C:\Python33\lib\site-packages\cx_Freeze\freezer.py", line 603, in Freeze

    self.finder = self._GetModuleFinder()
  File "C:\Python33\lib\site-packages\cx_Freeze\freezer.py", line 343, in _GetMo
duleFinder
    finder.IncludeModule(name)
  File "C:\Python33\lib\site-packages\cx_Freeze\finder.py", line 678, in Include
Module
    namespace = namespace)
  File "C:\Python33\lib\site-packages\cx_Freeze\finder.py", line 386, in _Import
Module
    raise ImportError("No module named %r" % name)
ImportError: No module named 'cherrypy.wsgiserver'
2014-07-25 12:44:15,797|ERROR: Command failed: 'C:\counterpartyd_build\env\Scrip
ts\python.exe "C:\counterpartyd_build\dist\_cxfreeze_setup.py" build -b "C:\coun
terpartyd_build\bin"'

Installing via pip doesn't help.

C:\counterpartyd_build>pip install cherrypy-wsgiserver
Requirement already satisfied (use --upgrade to upgrade): cherrypy-wsgiserver in
 c:\users\rippler\appdata\local\enthought\canopy\user\lib\site-packages
Cleaning up...

A workaround is to install a binary build of cherrypy-wsgiserver for Windows that corresponds to your version and architecture of Python and then continue.

What are the current restriction to support only Ubuntu?

I would like to help if is posible to support Debian.

[solved/duplicate] counterpartyd graceful shutdown command needed

Either it's not documented or not available.

sign bootstrap files with pgp keys

to increase security
on bootstrap.counterparty.co

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

sudo sv start counterpartyd - doesn't work if service not set to autostart (= not created)

CPD 9.49.2 on U14.04 (and Windows 7).

If I answer that I don't want counterpartyd to start automatically, sudo sv start counterpartyd doesn't work. I thought that was normal (because I never answer with y), but the docs do not explain this.

2014-12-20 03:05:54,960|DEBUG: RUNNING COMMAND: ln -sf /home/rippler/counterpartyd_build/run.py /usr/local/bin/counterpartyd
Start counterpartyd automatically on system startup? (y/n): n
2014-12-20 03:09:03,943|INFO: Creating new configuration file at: /home/rippler/.config/counterpartyd/counterpartyd.conf
2014-12-20 03:09:03,944|INFO: NOTE: counterpartyd config file has been created at '/home/rippler/.config/counterpartyd/counterpartyd.conf'
2014-12-20 03:09:03,945|INFO: SETUP DONE. (It's time to kick ass, and chew bubblegum... and I'm all outta gum.)
rippler@ubuntu:~/counterpartyd_build$ sudo sv start counterpartyd
fail: counterpartyd: unable to change to service directory: file does not exist
rippler@ubuntu:~/counterpartyd_build$

I wanted to check what happens if I answer that I want counterpartyd to start automatically:

2014-12-20 03:13:04,091|DEBUG: RUNNING COMMAND: ln -sf /home/rippler/counterpartyd_build/run.py /usr/local/bin/counterpartyd
Start counterpartyd automatically on system startup? (y/n): y
2014-12-20 03:13:13,445|INFO: Setting up init scripts...
2014-12-20 03:13:13,446|DEBUG: RUNNING COMMAND: rm -f /etc/init/counterpartyd.conf /etc/init/counterpartyd.conf.override
2014-12-20 03:13:13,449|DEBUG: RUNNING COMMAND: cp -dRf --preserve=mode /home/rippler/counterpartyd_build/dist/linux/runit/counterpartyd /etc/sv/
2014-12-20 03:13:13,453|DEBUG: RUNNING COMMAND: touch /etc/sv/counterpartyd/down
2014-12-20 03:13:13,458|DEBUG: RUNNING COMMAND: ln -sf /etc/sv/counterpartyd /etc/service/
2014-12-20 03:13:13,461|DEBUG: RUNNING COMMAND: rm -f /etc/init/counterpartyd-testnet.conf /etc/init/counterpartyd-testnet.conf.override
2014-12-20 03:13:13,463|DEBUG: RUNNING COMMAND: cp -dRf --preserve=mode /home/rippler/counterpartyd_build/dist/linux/runit/counterpartyd-testnet /etc/sv/
2014-12-20 03:13:13,466|DEBUG: RUNNING COMMAND: touch /etc/sv/counterpartyd-testnet/down
2014-12-20 03:13:13,469|DEBUG: RUNNING COMMAND: rm -f /etc/service/counterpartyd-testnet
2014-12-20 03:13:13,473|DEBUG: RUNNING COMMAND: rm -f /etc/init/counterblockd.conf /etc/init/counterblockd.conf.override
2014-12-20 03:13:13,476|DEBUG: RUNNING COMMAND: cp -dRf --preserve=mode /home/rippler/counterpartyd_build/dist/linux/runit/counterblockd /etc/sv/
2014-12-20 03:13:13,480|DEBUG: RUNNING COMMAND: touch /etc/sv/counterblockd/down
2014-12-20 03:13:13,483|DEBUG: RUNNING COMMAND: rm -f /etc/service/counterblockd
2014-12-20 03:13:13,486|DEBUG: RUNNING COMMAND: rm -f /etc/init/counterblockd-testnet.conf /etc/init/counterblockd-testnet.conf.override
2014-12-20 03:13:13,488|DEBUG: RUNNING COMMAND: cp -dRf --preserve=mode /home/rippler/counterpartyd_build/dist/linux/runit/counterblockd-testnet /etc/sv/
2014-12-20 03:13:13,492|DEBUG: RUNNING COMMAND: touch /etc/sv/counterblockd-testnet/down
2014-12-20 03:13:13,494|DEBUG: RUNNING COMMAND: rm -f /etc/service/counterblockd-testnet
2014-12-20 03:13:13,497|DEBUG: RUNNING COMMAND: sed -ri "s/USER=xcpd/USER=rippler/g" /etc/service/counterpartyd/run
2014-12-20 03:13:13,501|DEBUG: RUNNING COMMAND: sed -ri "s/USER_HOME=\/home\/xcp/USER_HOME=\/home\/rippler/g" /etc/service/counterpartyd/run
2014-12-20 03:13:13,507|INFO: counterpartyd config file already exists at: '/home/rippler/.config/counterpartyd/counterpartyd.conf'
2014-12-20 03:13:13,508|INFO: SETUP DONE. (It's time to kick ass, and chew bubblegum... and I'm all outta gum.)
rippler@ubuntu:~/counterpartyd_build$ sudo sv start counterpartyd
ok: run: counterpartyd: (pid 10871) 0s, normally down

• If you answer y, service is created, otherwise it's not
• sudo sv start counterpartyd should not be used unless you answered y

Add release tags

If there's a problem with the current master or develop code, it's not easy to get a previous release.
https://github.com/CounterpartyXCP/counterpartyd_build/releases
Could they occasionally be tagged so that people can easily get one of earlier releases?

auto update counterpartyd on script run by default (via git)

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

setup.py is broken

The first error is this:

/bin/sh: 1: cd: can't cd to /home/xcp/counterpartyd_build/dist/counterpartyd

Then if this is created manually, it complains about missing pip-requirements.txt

Could not open requirements file: [Errno 2] No such file or directory: '/home/xcp/counterpartyd_build/dist/counterpartyd/pip-requirements.txt'
Storing debug log for failure in /home/xcp/.pip/pip.log

If that is manually copied to that directory, there's more errors.
Maybe there's a recursive copy command that was dropped from a recent release?

Signed tags (and commits?)!

Bug: (Windows) counterpartyD startup file

Upon completion of the install the counterparty installer asks if you want to create a startup file.

The python version is incorrect in C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\run_counterpartyd

It defaults to version 3.2 and not the current version

C:\Python32\python.exe C:\counterpartyd_build\run.py server

Federated node build from Master fails at "dpkg -i /tmp/armory.deb"

Ubuntu 14.04 x64, building from Master, no email, no security hardening

 sudo python3 setup_federated_node.py
...

2014-08-23 15:36:31,831|DEBUG: RUNNING COMMAND: dpkg -i /tmp/armory.deb
(Reading database ... 87537 files and directories currently installed.)
Preparing to unpack /tmp/armory.deb ...
Unpacking armory (0.92.1-1) over (0.92.1-1) ...
xdg-icon-resource: No writable system icon directory found.
dpkg: warning: subprocess old post-removal script returned error exit status 3
dpkg: trying script from the new package instead ...
xdg-icon-resource: No writable system icon directory found.
dpkg: error processing archive /tmp/armory.deb (--install):
 subprocess new post-removal script returned error exit status 3
xdg-icon-resource: No writable system icon directory found.
dpkg: error while cleaning up:
 subprocess new post-removal script returned error exit status 3
Processing triggers for mime-support (3.54ubuntu1) ...
Errors were encountered while processing:
 /tmp/armory.deb
2014-08-23 15:36:33,532|ERROR: Command failed: 'dpkg -i /tmp/armory.deb'

move to using docker for builds

create Dockerfiles and build_docker.py script that will:

• take a command line arg (or prompt) for the role to build
  • multiple docker files, one for each build config...e.g. counterwallet_simple, counterwallet_full, counterpartyd_only, etc
  • all docker files build off of https://github.com/phusion/baseimage-docker
• execute the build process to generate a docker image for the specified role
  • the Dockerfile will download and call setup_federated_node.py itself, to build the federated node in the docker image with the approprite command line options
    (we will use this all with automated builds on dockerhub, and each build variant will be a tag)

Images I'd be planning:

• counterwallet_bitcoind: Uses bitcoind, no insight (uses blockr.io), and auto downloads the bitcoind blockchain data and counterpartyd and counterblock databases on first boot via bittorrent or s3 site
• counterwallet_simple: Uses pyrpcwallet (and blockr.io), no insight (uses blockr.io), and auto downloads the counterpartyd and counterblock databases on first boot via bittorrent or s3 site
• counterpartyd_simple: Same as counterwallet_simple, but only runs counterpartyd, with open port access and rpc/1234 username/password on rpc services by default...allows easy dev counterpartyd access

Future possible images:

• counterwallet_full: Uses bitcoind and insight. Blockchain data must manually download
• vending_machine
• autobtcescrow_agent
• whatever else

More notes:

• build logic for everything except ubuntu 14.04 can then be ripped out (I'm thinking) of setup_federated_node.py since other OSes can just use the docker image. alert users to this because this is disruptive though
• ubuntu 14.04-based systems can still use setup_federated_node.py directly. this is especially important for production servers where we want everything to run on bare metal, and the hardware is dedicated (e.g. counterwallet.co)
• also, include plenty of documentation for using the docker images on windows, macos, and other versions of linux

Docker resources:

• https://gist.github.com/wsargent/7049221
• http://crosbymichael.com/dockerfile-best-practices-take-2.html

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Issue: Check for openssl during install

Please see https://forums.counterparty.io/discussion/748/counterpartyd-v9-45-0-released-how-to-update#latest

Unable to run counterpartyd if the openssl dlls are not already available under the windows folders.

On Ubuntu 14.04 LTS installer for v9.31.0 cannot detect if cherrypy is installed

Today I helped someone install counterpartyd and upon startup counterpartyd complained there's no cherrypy.

Apparently it was because cherrypy was installed via easy_install, so it was present but not really (or not the right version).

A workaround was to install another version using apt-get (there are 3 python-cherrypy packages in Ubuntu, one of them is right), then run counterpartyd server started fine.

I'm not sure if this is a bug, maybe the installer should check for the right version of cherrpy.
Either way, my secondary purpose is to ultimately get this to the list of closed issues for others.

Bootstrap DB download activity or progress indicator

I wanted to submit a PR but I can't figure it out...

Now the DB is large and download speed is usually about 230K so it takes 20-30 mins which is quite a long time. It'd be nice to have a simple progress indicator (can be a very simple activity, not progress (% complete), indicator).
Yesterday my DB download somehow got stuck and I waited 40 mins before I checked network traffic and realized there was nothing going on.

Issue with bitcore checksum in Federated Node build

counterpartyd server - permission denied: '/home/xcp/.config/counterpartyd/counterpartyd.log'

xcp@ubuntu:~/.config/counterpartyd$ counterpartyd --config-file=~/.config/counterpartyd/counterpartyd.conf --data-dir=~/.config/counterpartyd --backend-rpc-user=rpc --backend-rpc-password=pass --rpc-password=pass --rpc-user=rpc server
Traceback (most recent call last):
  File "/home/xcp/counterpartyd_build/dist/counterpartyd/counterpartyd.py", line 733, in <module>
    fileh = logging.handlers.RotatingFileHandler(config.LOG, maxBytes=max_log_size, backupCount=5)
  File "/usr/lib/python3.4/logging/handlers.py", line 150, in __init__
    BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
  File "/usr/lib/python3.4/logging/handlers.py", line 57, in __init__
    logging.FileHandler.__init__(self, filename, mode, encoding, delay)
  File "/usr/lib/python3.4/logging/__init__.py", line 992, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/lib/python3.4/logging/__init__.py", line 1016, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: '/home/xcp/.config/counterpartyd/counterpartyd.log'

xcp@ubuntu:~/.config/counterpartyd$ sudo counterpartyd --config-file=~/.config/counterpartyd/counterpartyd.conf --data-dir=~/.config/counterpartyd --backend-rpc-user=rpc --backend-rpc-password=pass --rpc-password=pass --rpc-user=rpc server
Please run this script as a non-root user.

xcp@ubuntu:~/.config/counterpartyd$ ll /home/xcp/.config/counterpartyd/counterpartyd.log
-rw-r--r-- 1 xcpd nogroup 5896491 Nov  5 04:17 /home/xcp/.config/counterpartyd/counterpartyd.log

Then:

xcp@ubuntu:~$ sudo chown -R xcp:xcp .config/
xcp@ubuntu:~$ counterpartyd --config-file=~/.config/counterpartyd/counterpartyd. conf --data-dir=~/.config/counterpartyd --backend-rpc-user=rpc --backend-rpc-pas sword=pass --rpc-password=pass --rpc-user=rpc server
Status: Connecting to database.
...

There's a lot of files in .config directory that are owned by root and nogroup.
Maybe I didn't set the right permissions, but the fact is counterpartyd doesn't seem to work unless default permissions are changed.

pip needs to be updated

pypa/pip@90b1179

Windows "build from source" procedure needs to be reviewed for Python 3.4.1

See this for new dependencies:
CounterpartyXCP/counterparty-core#260

Are both 32-bit and 64-bit Python builds going to remain supported?

Issue with bitcoinjs-min.js in federated node setup

OS: Ubuntu 14.04
Counterparty_build: pulled from dev branch of git.

cheksum ok: src/vendors/jqlog/jquery.jqlog-1.3.min.js
Fatal error: Invalid checksum: src/vendors/bitcoinjs-lib/bitcoinjs-min.js
2014-05-07 12:06:56,019|ERROR: Command failed: 'cd ~xcp/counterwallet && grunt build'
rippler@ubuntu:~/counterpartyd_build$ md5sum src/vendors/bitcoinjs-lib/bitcoinjs-min.js
md5sum: src/vendors/bitcoinjs-lib/bitcoinjs-min.js: No such file or directory
rippler@ubuntu:~/counterpartyd_build$ find . -name "bitcoinjs-min.js"

I couldn't figure out where this file is supposed to come from and where is its checksum, so I devised a potential temporary workaround for those who can't wait to get started:

1. Fetch
   http://cdn.bitcoinjs.org/libs/bitcoinjs-lib/0.1.0/bitcoinjs-min.js
sudo chown root:root bitcoinjs-min.js
sudo cp -p bitcoinjs-min.js /home/xcp/counterwallet/src/vendors/bitcoinjs-lib/
2. Cheat
   sudo vim /home/xcp/counterwallet/src/index.html
3. Rebuild
   Done, without errors.

Fed Node script assumes xcp's home is /home/xcp

... which is reasonable, but the problem is if you move xcp to a large partition you need to mount it to /home/xcp otherwise the installer can fill out your system disk (or home partition).
Not a bug per se, but should be documented.

setup_federated_node improvements pre-docker

• switch federated node to use ruinit scripts instead of upstart scripts
  • add autoupdate logic from git to counterpartyd startup script (e.g. on startup, before starting counterpartyd, IF we are running off of master, update from git master)
• add ability to run setup_federated_node.py non-interactively (by providing question answers via command line)
• add ability for setup_federated_node.py to set up to use pyrpcwallet instead of bitcoind
  • pyrpcwallet will need to be updated to use blockr/chain.so(?) and support testnet

Automatically bootstrap DB with bootstrap files

Doesn't play well with new lockfile.

See CounterpartyXCP/counterparty-core#336 and CounterpartyXCP/counterparty-core#371.

Fed Node update: rollbar - mismatch version declared in the json (1.0.0-rc.8)

I don't know if this should be reported here, but ...
While updating Federated Note on U14.04 from 9.41.0 I noticed some rollbar mismatches.
Just in case other info may be useful I'll leave it here.

bower sprintf#d5cfad02b3db7a472544b1d9e467fcc126e453a9            invalid-meta sprintf is missing "ignore" entry in bower.json
bower sprintf#d5cfad02b3db7a472544b1d9e467fcc126e453a9                resolved git://github.com/alexei/sprintf.js.git#d5cfad02b3
bower counterwebdeps#7564435e9766c48b2e06f8e43611eeed4bbb6ba5     invalid-meta counterwebdeps is missing "main" entry in bower.json
bower counterwebdeps#7564435e9766c48b2e06f8e43611eeed4bbb6ba5     invalid-meta counterwebdeps is missing "ignore" entry in bower.jso                                                          n
bower counterwebdeps#7564435e9766c48b2e06f8e43611eeed4bbb6ba5         resolved https://github.com/CounterpartyXCP/counterwebdeps.git                                                          #7564435e97
bower rollbar#1.1.7                                                   progress received 1.4MB
bower rollbar#1.1.7                                                   progress received 1.7MB
bower rollbar#1.1.7                                                    extract archive.tar.gz
bower rollbar#1.1.7                                                   mismatch Version declared in the json (1.0.0-rc.8) is different than the resolved one (1.1.7)
bower rollbar#1.1.7                                               invalid-meta rollbar is missing "ignore" entry in bower.json
bower rollbar#1.1.7                                                   resolved git://github.com/rollbar/rollbar.js.git#1.1.7
bower smartadmin#c7049658057b26747833f18085255a795f10b581         invalid-meta smartadmin is missing "main" entry in bower.json
bower smartadmin#c7049658057b26747833f18085255a795f10b581         invalid-meta smartadmin is missing "ignore" entry in bower.json
bower smartadmin#c7049658057b26747833f18085255a795f10b581             resolved https://github.com/CounterpartyXCP/countersmartadmin.                   

Missing attribute 'CLIENT_VERSION' while creating binary build of counterparty v9.34.0 on Windows

C:\counterpartyd_build>python -V
Python 3.3.5

C:\counterpartyd_build>counterpartyd -V

C:\counterpartyd_build>echo off
counterpartyd v9.34.0

Towards the end of python setup.py build, it throws this error:

Output: "C:\counterpartyd_build\dist\windows\counterpartyd_install.exe"
Install: 7 pages (448 bytes), 4 sections (4192 bytes), 845 instructions (23660 b
ytes), 247 strings (5651 bytes), 1 language table (266 bytes).
Uninstall: 3 pages (192 bytes),
1 section (1048 bytes), 385 instructions (10780 bytes), 116 strings (1819 bytes)
, 1 language table (218 bytes).
Datablock optimizer saved 9072 bytes (~0.1%).

Using zlib compression.

EXE header size:               36352 / 35840 bytes
Install code:                   5844 / 34545 bytes
Install data:                7184244 / 12946563 bytes
Uninstall code+data:            2306 / 2807 bytes
CRC (0x5674AD3B):                  4 / 4 bytes

Total size:                  7228750 / 13019759 bytes (55.5%)

1 warning:
  uninstall function "un.EnvVarUpdate" not referenced - zeroing code (179-377) o
ut

Traceback (most recent call last):
  File "setup.py", line 555, in <module>
    main()
  File "setup.py", line 536, in main
    do_build(paths, with_counterblockd)
  File "setup.py", line 466, in do_build
    installer_dest = os.path.join(paths['bin_path'], "counterpartyd-v%s-%s_insta
ll.exe" % (config.CLIENT_VERSION, arch))
AttributeError: 'module' object has no attribute 'CLIENT_VERSION'

counterpartyd DB error after Federated Node rebuild

Was 9.42.0, update to 9.45.0 first failed, then I rebuilt, however while bitcoind is okay, I'm getting this error with counterpartyd:

Status: Connecting to database.
Traceback (most recent call last):
  File "/home/xcp/counterpartyd_build/dist/counterpartyd/counterpartyd.py", line 755, in <module>
    db = util.connect_to_db()
  File "/home/xcp/counterpartyd_build/dist/counterpartyd/lib/util.py", line 322, in connect_to_db
    cursor.execute('''PRAGMA journal_mode = WAL''')
  File "src/cursor.c", line 990, in APSWCursor_execute.sqlite3_prepare
  File "src/statementcache.c", line 386, in sqlite3_prepare
apsw.CantOpenError: CantOpenError: unable to open database file
xcp@ubuntu:~$

counterpartyd isntall fails due to python-bitcoinlib 0.3.0

Setup

Ubuntu 14.04 x86_64
Linux digital-oceon-vm 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

The failure

Main error: Could not find a version that satisfies the requirement python-bitcoinlib==0.3.0:

Downloading/unpacking python-bitcoinlib==0.3.0 (from -r /home/btc/counterpartyd_build/dist/counterpartyd/pip-requirements.txt (line 34))
  Could not find a version that satisfies the requirement python-bitcoinlib==0.3.0 (from -r /home/btc/counterpartyd_build/dist/counterpartyd/pip-requirements.txt (line 34)) (from versions: 0.1.1, 0.2.0, 0.2.1)
  Some externally hosted files were ignored (use --allow-external to allow).
Cleaning up...
No distributions matching the version for python-bitcoinlib==0.3.0 (from -r /home/btc/counterpartyd_build/dist/counterpartyd/pip-requirements.txt (line 34))
Storing debug log for failure in /home/btc/.pip/pip.log
2014-12-27 16:22:36,706|ERROR: Command failed: '/home/btc/counterpartyd_build/env/bin/pip install -r /home/btc/counterpartyd_build/dist/counterpartyd/pip-requirements.txt'

Installation log

http://pastebin.com/q4Dd1V4v

can't say "no" to "run as on startup" question during installation

This is what I got:

2014-02-28 00:58:28,788|DEBUG: RUNNING COMMAND: sudo ln -sf /home/guy/tmp/ripple/counterpartyd_build/run.py /usr/local/bin/counterpartyd
Start counterpartyd automatically on system startup? (y/n): n
Traceback (most recent call last):
File "./setup.py", line 505, in <module>
    main()
File "./setup.py", line 497, in main
    setup_startup(paths, run_as_user, with_counterwalletd)
File "./setup.py", line 302, in setup_startup
    start_choice = input("Start counterpartyd automatically on system startup? (y/n): ")
File "<string>", line 1, in <module>
NameError: name 'n' is not defined