xeroc/python-graphenelib

Python3 library for Graphene-based blockchains

Python Library for Graphene

Current build status

---

Documentation

Visit the pygraphenelib website for in depth documentation on this Python library.

Installation

Install with pip3:

$ sudo apt-get install libffi-dev libssl-dev python-dev
$ pip3 install graphenelib

Manual installation:

$ git clone https://github.com/xeroc/python-graphenelib/
$ cd python-graphenelib
$ python3 setup.py install --user

Upgrade

$ pip install --user --upgrade graphenelib

Contributing

python-bitshares welcomes contributions from anyone and everyone. Please see our guidelines for contributing and the code of conduct.

Discussion and Developers

Discussions around development and use of this library can be found in a dedicated Telegram Channel

License

A copy of the license is available in the repository's LICENSE file.

python-graphenelib's Issues

Block['id'] does not exist if string passed as 'block number' argument

Issue

If you construct a Block object and pass a string as the block argument, the base class constructor BlockchainObject.__init__() does not set the 'id' attribute. This causes the Block constructor to behave differently when passing the block number argument as an integer VS a string.

In my case, this issue cases my BitShares blockchain explorer to miss caching some blocks.

CurrencyLayer API

Getting some strange results trying to setup this API, eg:
Price for asset EUR has change from 0.003400 to 1607.205657 (47276524.082367%)! Do you want to continue? [Y/n] n

Keys Config

################################################################################
# Feed Keys
################################################################################
#~ # Acquire a OpenExchange API Key from:
#~ # https://openexchangerates.org
#~ OpenExchangeApiKey = "mykey"

# Acquire a CurrencyLayer API Key from:
# https://currencylayer.com
CurrencyLayerApiKey = "mykey"

Sources Config

                    "sources" : [  # Required Exchanges for FIAT
                                 "btcavg",    # To get from BTC into USD/CNY/EUR
                                 "yahoo",     # To get from USD/CNY/EUR into other FIAT currencies
                                 "quandl",
                                 "google",
                                 "currencylayer",
                                 # BTC/BTS exchanges (include BTC/* if available)
                                 "poloniex",
                                 "bittrex",
                                 "btc38",
                                 "yunbi",
                                 "bitshares",
                                 # ## BTC/* exchanges
                                 # "okcoin",   # no trading-fees
                                 # "btcchina", # no trading-fees
                                 # "huobi",    # no trading-fees
                                 ],

FeedSources Config

feedSources["currencylayer"] = feedsources.CurrencyLayer(api_key=CurrencyLayerApiKey,
                                                                  free_subscription=True,
                                                                  allowFailure=True,
                                                                  quotes=["BTC", "CNY", "EUR", "GBP", "JPY"], # more available
                                                                  bases=["USD"]) # only USD with free subscription

CurrencyLayer also has XAU and XAG. Can this be added?

Output

bts_feed.sh 
(poloniex)(okcoin)(currencylayer)(yahoo)(coincap)(btcchina)(bitshares)(google)(quandl)(btcavg)(yunbi)(btc38)(bittrex)(coinmarketcap)(huobi).
Google We encountered an error loading live data. Trying to recover from cache! (
Error fetching results from Google! (division by zero))
........
Btc38 We encountered an error loading live data. Trying to recover from cache! (
Error fetching results from Btc38! (Expecting value: line 1 column 1 (char 0)))
......Fetching data from wallet...
[Warning] Only a single source for the CNY price could be identified
[Warning] Only a single source for the CNY price could be identified
Asset EUR shows high variance in fetched prices!
Asset EUR shows high variance in fetched prices!
Asset USD shows high variance in fetched prices!
Asset USD shows high variance in fetched prices!
Asset KRW shows high variance in fetched prices!
Asset KRW shows high variance in fetched prices!
Asset USD shows high variance in fetched prices!
Asset USD shows high variance in fetched prices!
Asset USD shows high variance in fetched prices!
Asset USD shows high variance in fetched prices!
BTC: New Feeds differs too much: 0.97986205% > 0.50000000%! Force updating!
SILVER: New Feeds differs too much: 1.20004751% > 0.50000000%! Force updating!
GOLD: New Feeds differs too much: 1.19925675% > 0.50000000%! Force updating!
TRY: New Feeds differs too much: 1.08972450% > 0.50000000%! Force updating!
SGD: New Feeds differs too much: 1.08360665% > 0.50000000%! Force updating!
HKD: New Feeds differs too much: 1.11903551% > 0.50000000%! Force updating!
NZD: New Feeds differs too much: 1.08019301% > 0.50000000%! Force updating!
MXN: New Feeds differs too much: 1.01760493% > 0.50000000%! Force updating!
CAD: New Feeds differs too much: 1.14250986% > 0.50000000%! Force updating!
CHF: New Feeds differs too much: 1.09792122% > 0.50000000%! Force updating!
AUD: New Feeds differs too much: 1.20703801% > 0.50000000%! Force updating!
GBP: New Feeds differs too much: 9.19652930% > 0.50000000%! Force updating!
JPY: New Feeds differs too much: 1.09084877% > 0.50000000%! Force updating!
Price for asset EUR has change from 0.003400 to 1607.205657 (47276524.082367%)! Do you want to continue? [Y/n] n
Price for asset USD has change from 0.003582 to 0.182824 (5003.948787%)! Do you want to continue? [Y/n] n
KRW: New Feeds differs too much: 0.94970559% > 0.50000000%! Force updating!
Price for asset TUSD has change from 0.003582 to 0.182824 (5003.948787%)! Do you want to continue? [Y/n] n
Price for asset CASH.USD has change from 0.003582 to 0.182824 (5003.948787%)! Do you want to continue? [Y/n] n
CASH.BTC: New Feeds differs too much: 0.97986205% > 0.50000000%! Force updating!
ARS: New Feeds differs too much: 1.11614093% > 0.50000000%! Force updating!
+----------+------------+---------------+-------------------------+----------------------------+------------------------+------------------+----------------------------+----------------------------+--------------------+---------+
|  asset   | feed quote |   new price   |           mean          |           median           |       wgt. avg.        |   wgt. std (#)   |         blockchain         |       my last price        |    last update     | publish |
+----------+------------+---------------+-------------------------+----------------------------+------------------------+------------------+----------------------------+----------------------------+--------------------+---------+
|   NZD    |    BTS     |   0.00520572  |   0.01014700 (-48.70%)  |    0.00534065 (-2.53%)     |  0.00520572 (+0.00%)   |    0.10% (96)    |    0.00520727 (-0.03%)     |    0.00515009 (+1.08%)     | 0:18:56.452804 ago |    X    |
|  SILVER  |    BTS     |   0.00022665  |   0.00127680 (-82.25%)  |    0.00023286 (-2.67%)     |  0.00022665 (+0.00%)   |    0.01% (10)    |    0.00021843 (+3.76%)     |    0.00022396 (+1.20%)     | 0:18:56.452987 ago |    X    |
|  ALTCAP  |    BTC     |   6.02019352  |   6.02073131 (-0.01%)   |    6.02073131 (-0.01%)     |  6.02019352 (+0.00%)   |    5.69% ( 2)    |    6.02661515 (-0.11%)     |    6.01854414 (+0.03%)     | 0:18:56.453110 ago |         |
|   GBP    |    BTS     |   0.00318309  |   0.00756291 (-57.91%)  |    0.00302424 (+5.25%)     |  0.00318309 (+0.00%)   |    0.10% (106)   |    0.00294758 (+7.99%)     |    0.00291501 (+9.20%)     | 0:18:56.453385 ago |    X    |
|   TUSD   |    BTS     |   0.18282425  |   2.04582338 (-91.06%)  |   0.00371346 (+4823.29%)   |  0.18282425 (+0.00%)   |  125.39% (100)   |   0.00361554 (+4956.62%)   |   0.00358202 (+5003.95%)   | 0:18:56.453504 ago |         |
|   CAD    |    BTS     |   0.00479158  |   0.00934227 (-48.71%)  |    0.00491693 (-2.55%)     |  0.00479158 (+0.00%)   |    0.09% (96)    |    0.00479309 (-0.03%)     |    0.00473746 (+1.14%)     | 0:18:56.453616 ago |    X    |
|   CNY    |    BTS     |   0.02570000  |   0.02570000 (+0.00%)   |    0.02570000 (+0.00%)     |  0.02570000 (+0.00%)   |    0.00% ( 1)    |    0.02570000 (+0.00%)     |    0.02570000 (+0.00%)     | 0:17:05.453729 ago |         |
|   JPY    |    BTS     |   0.42379437  |   0.87139527 (-51.37%)  |    0.43467145 (-2.50%)     |  0.42379437 (+0.00%)   |    7.82% (166)   |    0.42607700 (-0.54%)     |    0.41922130 (+1.09%)     | 0:18:56.453837 ago |    X    |
|   MXN    |    BTS     |   0.07691696  |   0.14984090 (-48.67%)  |    0.07886905 (-2.48%)     |  0.07691696 (+0.00%)   |    1.41% (96)    |    0.07697050 (-0.07%)     |    0.07614213 (+1.02%)     | 0:18:56.453947 ago |    X    |
|   SGD    |    BTS     |   0.00521145  |   0.01016193 (-48.72%)  |    0.00534874 (-2.57%)     |  0.00521145 (+0.00%)   |    0.10% (96)    |    0.00521627 (-0.09%)     |    0.00515558 (+1.08%)     | 0:18:56.454057 ago |    X    |
|   USD    |    BTS     |   0.18282425  |   2.04582338 (-91.06%)  |   0.00371346 (+4823.29%)   |  0.18282425 (+0.00%)   |  125.39% (100)   |   0.00361554 (+4956.62%)   |   0.00358202 (+5003.95%)   | 0:18:56.454167 ago |         |
| CASH.BTC |    BTS     |   0.00000399  |   0.00000402 (-0.86%)   |    0.00000399 (+0.00%)     |  0.00000399 (+0.00%)   |    0.00% ( 3)    |    0.00000398 (+0.38%)     |    0.00000395 (+0.98%)     | 1:18:53.454274 ago |    X    |
|   AUD    |    BTS     |   0.00496286  |   0.00967642 (-48.71%)  |    0.00509593 (-2.61%)     |  0.00496286 (+0.00%)   |    0.09% (96)    |    0.00496018 (+0.05%)     |    0.00490367 (+1.21%)     | 0:18:56.454390 ago |    X    |
|   GOLD   |    BTS     |   0.00000314  |   0.00001771 (-82.25%)  |    0.00000323 (-2.67%)     |  0.00000314 (+0.00%)   |    0.00% (10)    |    0.00000307 (+2.47%)     |    0.00000311 (+1.20%)     | 0:18:56.454502 ago |    X    |
|   CHF    |    BTS     |   0.00368531  |   0.00718512 (-48.71%)  |    0.00378140 (-2.54%)     |  0.00368531 (+0.00%)   |    0.07% (96)    |    0.00368717 (-0.05%)     |    0.00364529 (+1.10%)     | 0:18:56.454614 ago |    X    |
|   KRW    |    BTS     |   4.35982718  |   8.57440599 (-49.15%)  |    4.46726625 (-2.41%)     |  4.35982718 (+0.00%)   |   82.80% (96)    |    4.47583000 (-2.59%)     |    4.31881119 (+0.95%)     | 0:18:56.454721 ago |    X    |
|   TRY    |    BTS     |   0.01319735  |   0.02573650 (-48.72%)  |    0.01354336 (-2.55%)     |  0.01319735 (+0.00%)   |    0.24% (96)    |    0.01321423 (-0.13%)     |    0.01305508 (+1.09%)     | 0:18:56.454830 ago |    X    |
| CASH.USD |    BTS     |   0.18282425  |   2.04582338 (-91.06%)  |   0.00371346 (+4823.29%)   |  0.18282425 (+0.00%)   |  125.39% (100)   |   0.00361151 (+4962.27%)   |   0.00358202 (+5003.95%)   | 0:18:56.454940 ago |         |
|   BTC    |    BTS     |   0.00000399  |   0.00000402 (-0.86%)   |    0.00000399 (+0.00%)     |  0.00000399 (+0.00%)   |    0.00% ( 3)    |    0.00000398 (+0.25%)     |    0.00000395 (+0.98%)     | 1:18:53.455049 ago |    X    |
|   ARS    |    BTS     |   0.05742601  |   0.11218609 (-48.81%)  |    0.05915520 (-2.92%)     |  0.05742601 (+0.00%)   |    1.06% (48)    |    0.05744799 (-0.04%)     |    0.05679213 (+1.12%)     | 0:18:56.455157 ago |    X    |
|   EUR    |    BTS     | 1607.20565743 | 18.04607372 (+8806.12%) | 0.00352730 (+45564666.01%) | 1607.20565743 (+0.00%) | 147353.76% (164) | 0.00343366 (+46807319.02%) | 0.00339958 (+47276524.08%) | 0:18:56.455265 ago |         |
|   HKD    |    BTS     |   0.02808064  |   0.05985779 (-53.09%)  |    0.02880878 (-2.53%)     |  0.02808064 (+0.00%)   |    0.52% (160)   |    0.02809660 (-0.06%)     |    0.02776988 (+1.12%)     | 0:18:56.455373 ago |    X    |
+----------+------------+---------------+-------------------------+----------------------------+------------------------+------------------+----------------------------+----------------------------+--------------------+---------+
Blamefile: /home/ihashfury/git/iPerky-python-graphenelib/scripts/pricefeeds/blame/12942498.json
Are you SURE you would like to publish this feed? [Y/n] n

Certificate validation is disabled for wss urls

https://github.com/xeroc/python-graphenelib/blob/master/grapheneapi/graphenewsrpc.py#L78

This disables certificate validation for secure (wss://) endpoints, rendering them useless at protecting against a man-in-the-middle attacker.

This is a high severity bug for a project such as this that people may be using to manage their assets.

Bitshares specifics in TransactionBuilder constructTx()

ops = self.add_required_fees(ops, asset_id=self.fee_asset_id)

This is bitshares specific as another graphene chains may not have "fee" in operations.

[Exchangebots] Cancelling an order doesn't remove it from the state

./scripts/exchangebots/strategies/basestrategy.py
Any strategy that inherits from base strategy will see a cancelled order as a filled order.

suggested solution: add

self.state["orders"][market].remove(ordered)

to the cancel actions in basestrategy.py

    def cancel_this_markets(self, side="both") :
        """ Cancel all orders in all markets of that are served by this
            bot.
            :param str side: cancel only "sell", "buy", "both" side(s) (default: "both")
            :return: number of cancelled orders
            :rtype: number
        """
        orders = self.dex.returnOpenOrders()
        numCanceled = 0
        for m in self.settings["markets"]:
            for o in orders[m]:
                if o["type"] is side or side is "both":
                    try :
                        print("Canceling %s" % o["orderNumber"])
                        self.dex.cancel(o["orderNumber"])
                        numCanceled += 1
                    except:
                        print("An error has occured when trying to cancel order %s!" % o["orderNumber"])
        return numCanceled

 def loadMarket(self, notify=True):
        """ Load the markets and compare the stored orders with the
            still open orders. Calls ``orderFilled(orderid)`` for orders no
            longer open (i.e. fully filled)
        """
        #: Load Open Orders for the markets and store them for later
        self.opened_orders = self.dex.returnOpenOrdersIds()

        #: Have orders been matched?
        old_orders = self.getState()["orders"]
        cur_orders = self.dex.returnOpenOrdersIds()
        for market in self.settings["markets"] :
            if market in old_orders:
                for orderid in old_orders[market] :
                    if orderid not in cur_orders[market] :
                        # Remove it from the state
                        self.state["orders"][market].remove(ordered)
                        # Execute orderFilled call
                        if notify :
                            self.orderFilled(ordered)

testing ALTCAP.X develop branch

Any further progress on this issue?

(poloniex)(okcoin)(coinmarketcap)(yunbi)(ccedk)(huobi)(bitshares)(btc38)(bittrex)(btcchina)(coincap)(btcavg)(yahoo)..
Fetched data from Btc38 is empty!
...........[Warning] No market route found for ALTCAP.X. Skipping price
Fetching data from wallet...
Warning: Asset ALTCAP.X has no derived price!

Discrepancy between order that should be placed and the order that is actually places (maker.py)

There seems to be an issue in: https://github.com/xeroc/python-graphenelib/blob/master/grapheneexchange/exchange.py

When a bot that implements basestrategy calls self.sell() this get's called:

print(" - Selling %f %s for %s @%f %s/%s" % (amount, quote, base, price, base, quote))
self.dex.sell(market, price, amount)

output:

 - Selling 0.000150 SILVER for BTS @3677.763500 BTS/SILVER
2016-04-17 14:23:41.220941 | Order: 1.7.232305 is 51.499% away from feed

order info:

{
    'expiration': '2016-04-24T14:22:40',
    'deferred_fee': 14676,
    'id': '1.7.232305',
    'sell_price': {
        'quote': {
            'asset_id': '1.3.0',
            'amount': 55166
        },
        'base': {
            'asset_id': '1.3.105',
            'amount': 1
        }
    },
    'for_sale': 1,
    'seller': '1.2.107199'
}

 {
    'total': 0.55166,
    'rate': 5516.6,
    'amount': 0.0001,
    'type': 'sell',
    'amount_to_sell': 0.0001,
    'orderNumber': '1.7.232305'
 }

trade type issue in returnTradeHistory() method

below is the return when I call returnTradeHistory("CNY_BTS") as account trans.bot
"CNY_BTS": [
{
"amount": 28.4,
"rate": 35.21126760563381,
"total": 1000.0000000000001,
"date": "2016-07-09T06:40:18",
"type": "buy"
},
{
"amount": 0.292,
"rate": 34.36229452054795,
"total": 10.03379,
"date": "2016-07-09T06:38:24",
"type": "sell"
}]

actually in both trade trans.bot pay CNY and receive BTS, the difference is that in the first trade(the top one) trans.bot trade as a maker, and in the second trade a taker, I check exchange.py and found the source data define "buy" or "sell" with the perspective of the taker.

so that means returnTradeHistory need not only check with "if f["op"]["pays"]["asset_id"] == m["base"] :", but also need to check whether f["op"]["account_id"] equal to the caller. and then can get the right trade type.

BTW, I found returnTradeHistory("BTS_CNY") does not work, possible to make this work?

Incompatible version error

I am installing it via pip for Pythin 3.5 and it gives error:

Could not find a version that satisfies the requirement graphenelibs (from versions: )
No matching distribution found for graphenelibs

CER is out by a factor of x 1000 for ALTCAP

SETTLEMENT PRICE – ALTCAP/BITBTC is correct but CER is out by a factor of x 1000 for ALTCAP

git describe --tags
0.4.8-10-g05eb76d

ValueError on empty ticker data

  File "/home/vvk/devel/bitshares/bitshares-pricefeed/bitshares_pricefeed/pricefeed.py", line 376, in compute_target_price
    ticker = Market("%s:%s" % (backing_symbol, symbol)).ticker()
  File "/home/vvk/.local/share/virtualenvs/bitshares-pricefeed-DCueNqwP/lib/python3.6/site-packages/bitshares/market.py", line 153, in ticker
    ticker["base_volume"], self["base"], blockchain_instance=self.blockchain
  File "/home/vvk/.local/share/virtualenvs/bitshares-pricefeed-DCueNqwP/lib/python3.6/site-packages/graphenecommon/instance.py", line 34, in __init__
    cls.__init__(self, *args, **kwargs)
  File "/home/vvk/.local/share/virtualenvs/bitshares-pricefeed-DCueNqwP/lib/python3.6/site-packages/graphenecommon/amount.py", line 132, in __init__
    self["amount"] = float(self["amount"])
ValueError: could not convert string to float: 

Raw ticker data, the asset is newly created:

{'time': '1970-01-01T00:00:00', 'base': '', 'quote': '', 'latest': '', 'lowest_ask': '', 'highest_bid': '', 'percent_change': '', 'base_volume': '', 'quote_volume': ''}

Error-prone amount comparison

To compare amounts asset comparison performed prior (assert):

    def __eq__(self, other):
        if isinstance(other, Amount):
            assert other["asset"] == self["asset"]
            return self["amount"] == other["amount"]

This means full dictionary comparison is happening, as self["asset"] is dict with asset data. If any asset data changed between Amounts instantiation, this will lead to error:

a = Amount('10 CNY')
b = Amount('10 CNY')
a['asset']['options']['core_exchange_rate']['base']['amount'] += 1
a == b

The solution is to compare only asset ids:
assert other["asset"]["id"] == self["asset"]["id"]

Windows installation broken due to secp256k1

secp256k1 package installation failing on Windows with error

'pkg-config' is required to install this package. Please see the README for details.

Codaone/DEXBot#535

Coinmarketcap API has changed

Coinmarketcap API has changed affecting ALTCAP smartcoin.

Error:

inmarketcapAltcap We encountered an error loading live data. Trying to recover from cache! (
Error fetching results from CoinmarketcapAltcap! (unorderable types: str() <= int()))

This also affects the CER calculation when only coincap.io is used for a feed price.

Bitshares specifics in TransactionBuilder.appendSigner()

  File "/home/vvk/devel/bitshares/python-graphenelib/graphenecommon/transactionbuilder.py", line 267, in appendSigner
    assert permission in ["active", "owner"], "Invalid permission"
AssertionError: Invalid permission

I suggest to use something like assert permission in self.permission_types to allow overriding from subclasses.

Blockchain.ops() performs two calls of 'get_dynamic_global_properties' for each get_block call

Example requests logged:

{'method': 'call', 'params': ['database_api', 'get_config', []], 'jsonrpc': '2.0', 'id': 1}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 2}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 3}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 4}
{'method': 'call', 'params': ['database_api', 'get_block', [4556896]], 'jsonrpc': '2.0', 'id': 5}
{'block_num': 4556896,
 'op': ['award',
        {'beneficiaries': [{'account': 'pom', 'weight': 500},
                           {'account': 'u68.pom', 'weight': 5000}],
         'custom_sequence': 0,
         'energy': 500,
         'initiator': 'u68.pom',
         'memo': 'Награда за сообщение в чате Telegram: CRYPTOKOT',
         'receiver': 'u117.pom'}],
 'timestamp': '2019-03-07T11:21:21'}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 6}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 7}
{'method': 'call', 'params': ['database_api', 'get_block', [4556897]], 'jsonrpc': '2.0', 'id': 8}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 9}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 10}
{'method': 'call', 'params': ['database_api', 'get_block', [4556898]], 'jsonrpc': '2.0', 'id': 11}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 12}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 13}
{'method': 'call', 'params': ['database_api', 'get_block', [4556899]], 'jsonrpc': '2.0', 'id': 14}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 15}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 16}
{'method': 'call', 'params': ['database_api', 'get_block', [4556900]], 'jsonrpc': '2.0', 'id': 17}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 18}
{'method': 'call', 'params': ['database_api', 'get_dynamic_global_properties', []], 'jsonrpc': '2.0', 'id': 19}
{'method': 'call', 'params': ['database_api', 'get_block', [4556901]], 'jsonrpc': '2.0', 'id': 20}

Stefan said he implement a solution in bexi https://github.com/blockchainbv/bexi/blob/master/bexi/blockchain_monitor/__init__.py

ModuleNotFoundError: No module named 'graphenebase.aes'

% python
Python 3.6.5 (default, Oct 30 2018, 16:28:34) 
[GCC 7.3.1 20180703] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bitshares.exceptions
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/bitshares-0.2.1-py3.6.egg/bitshares/__init__.py", line 1, in <module>
    from .bitshares import BitShares
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/bitshares-0.2.1-py3.6.egg/bitshares/bitshares.py", line 7, in <module>
    from .storage import get_default_config_store
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/bitshares-0.2.1-py3.6.egg/bitshares/storage.py", line 1, in <module>
    from graphenestorage import (
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/graphenelib-1.0.1-py3.6.egg/graphenestorage/__init__.py", line 2, in <module>
    from .base import (
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/graphenelib-1.0.1-py3.6.egg/graphenestorage/base.py", line 3, in <module>
    from .masterpassword import MasterPassword
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/graphenelib-1.0.1-py3.6.egg/graphenestorage/masterpassword.py", line 8, in <module>
    from graphenebase.aes import AESCipher
ModuleNotFoundError: No module named 'graphenebase.aes'

Using python-bitshares without writing anything to disk

Background:
I'm running python-bitshares on Google AppEngine, which doesn't allow writes to disk, causing exceptions because python-bitshares attempt to write the sqlite database to disk. This happens, even if BitShares() is initialized with keys: bitshares = BitShares(node=mynode, keys=[mykey])

Root Cause:
This line: self.config = kwargs.get("config_store", SqliteConfigurationStore(**kwargs)) (reference) is executed when a new BitShares() instance is created, resulting in disk writes.

This happens even if config_store is set while creating a new BitShares() instance like below:

from graphenestorage import InRamConfigurationStore
bitshares = BitShares(node=mynode, keys=[mykey], config_store=InRamConfigurationStore())

I suspect that kwargs.get() is causing this issue. For some reason it will call the default value even if another value is parsed to it (in this case SqliteConfigurationStore()). Once SqliteConfigurationStore() is called a folder will be created on disk (reference)

I'm not sure what the best way to fix this, I went with a quick hack:

bitshares = BitShares(node=node, keys=[app.config["WIF"]], config_store="ram") 

and then patched sqlite.py (adding the first line below):

        if kwargs.get("config_store", None) is None:
            if os.path.isdir(data_dir):  # pragma: no cover
                print("checking folder")
                return
            else:  # pragma: no cover
                print("creating folder")
                os.makedirs(data_dir)

Problem with bip38 (AssertionError) "Flagbyte has to be 0xc0"

Hello!
Please help. I am creating a bip38 wallet on the site bitaddress org.
For example:
Encrypted Private Key:
6PnWr5jA55QRpzjBzNF2wyPrpLhuihQu7WTgpzDgkyN8io3gXgX5cmfteq
Passphrase: 123123

Use this python code with graphenelib for decrypt:

from graphenebase.account import PrivateKey
from graphenebase.bip38 import encrypt, decrypt
dec = format(decrypt("6PnWr5jA55QRpzjBzNF2wyPrpLhuihQu7WTgpzDgkyN8io3gXgX5cmfteq", "123123"), "wif")
print(dec)

And get the error:
\Python\Python39\lib\site-packages\graphenebase\bip38.py", line 99, in decrypt
assert flagbyte == b"\xc0", "Flagbyte has to be 0xc0"
AssertionError: Flagbyte has to be 0xc0
Please help, how to fix.

CASH.USD

Having trouble feeding CASH.USD

git describe --tags
0.3.7-17-g83612f9

TCNY and TUSD are working correctly but the script skips CASH.USD

                #
                # As requested by the issuer, the squeeze ratio should be
                #100.1%
                "CASH.USD" : {
                    "maximum_short_squeeze_ratio"   : 1001,
                },
                "TCNY" : {
                    "maximum_short_squeeze_ratio"   : 1001,
                },
                "TUSD" : {
                    "maximum_short_squeeze_ratio"   : 1001
                },
               }

# Other assets that are derived or something else.
# Currently available:
#
#    "sameas" : x     # uses the same asset price as a MPA above
#
# Note:
#  The usual asset specific parameters have to be set in "asset_config",
#  otherwise they will be ignored!
secondary_mpas = {"CASH.USD" : {"sameas" : "USD"},
                  "TCNY" : {"sameas" : "CNY"},
                  "TUSD" : {"sameas" : "USD"}
                  }

################################################################################

graphenecommon.wallet.Wallet.getAccounts() uses deprecated method getAllAccounts()

accounts.extend(self.getAllAccounts(pubkey))

Need to rewrite the method to use getAccountsFromPublicKey()

Account Create and TESTNET

When adding

operations.Account_create(**op)

that contains a prefix flag into ProposalBuilder, it gets recreated when broadcasting (through self.constructTx() in TransactionBuilder), but at that point without the prefix, leading to fallback to "BTS" as prefix

issue in deep_eq

some data element are not in correct format while comparing, in the screenshot, while compare transactions, in the data got from block the min_to_receive['amount'] should be int but actually it is string, and the result is that when you submit an order with returnID=True but you cannot got the orderid

open orders

@xeroc
which api is to return the open order of myself, GrapheneExchange.returnOpenOrders()?is it open-source,seems i can not find it in this respository

pycrypto dependence missing?

with a fresh install, (pip3 install uptick), user can't run uptick until manual installation of pycrypto. seems missing depends in setup.py but I am not sure.

Modules docs

For each module there is file like graphenebase.memo.rst. I suggest to auto-generate such files via sphinxcontrib-apidoc module, would you not mind if I add this into my further PR?

Wishlist for API/API Design/documentation?

Hi xeroc,

API Wishlist?

just curious what your wishlist for the client/WS API is - perhaps I can add them to the witness_node or cli_wallet.

Library Architecture?

Also, related - what do you envision as the overall architecture of the library, in the end? Would you want to create an even higher level abstraction of the current API?

Documentation?

It could be good to document how the library interacts with everything. This is my understanding so far:

1. the api class can connect to cli_wallet and implements all RPC commands
2. the ws api class is a subclass of api, connects to witness_node via authenticated ws and also via unauthenticated RPC commands
3. the wsprotocol implements asynchronous message handling.

Regarding documentation:

awesome features:

1. the wsprotocol has a class attribute of objectMap, which is a dictionary of every object you have looked in to - and it is updated every block as needed! Word of caution: orders (1.7.*) are deleted from the blockchain when they are filled, but they are not deleted from the objectMap as no notification is ever sent by witness_node when they are deleted! (cryptonomex/graphene#488) Also, when you use getObject or getObjectcb, the library checks objectMap first - saving extra calls!

Some things I learned:

1. objectMap is shared among all instances of the wsprotocol class, so even though the ws api only stores the class (not the instance), it will have access to objectMap. Instances are generated by autobahn's websocket factory.
2. the self.api_ids is not shared among all instances of the wsprotocol class. This causes problems if you try to connect to a restricted API in anything except a subclass of your wsprotocol
3. That said, you can "trick" python into giving you the actual instance of the protocol if you set a callback to function in another class. This causes both your new class and the protocol to be passed as the first and second arguments to the method in the new class, thus giving you access to self.api_ids

A complex use case

I need something with both cli_wallet and witness_node interface, and I want to get the actual protocol passed back to me. I can do something like this:

1. Create a new class

class MarketBalanceBot :
    def __init__(self, wsapi, walletapi) :
        self.wsapi=wsapi
        self.walletapi=walletapi
        self.walletapi.unlock(config.password)
            ## Set Callbacks for object changes
        self.wsapi.setObjectCallbacks({"2.1.0" : self.oneNewBlock})
    def onNewBlock(self,proto,data):
        print("I'm doing this just once, as I have the real protocol!")
        proto.database_callbacks_ids.pop("2.1.0")
        print("How I can see the api ids with the real protocol:")
        print(proto.api_ids)
        print("But my wsapi class doesn't know the api ids!")
        try:
            print(self.wsapi.proto.api_ids)
        except:
            print("error:", sys.exc_info()[0])

if __name__ == '__main__':
    wsapi      = GrapheneWebsocket(config.IP,config.WSPORT,"","", GrapheneWebsocketProtocol)
    clientrpc=GrapheneAPI(config.IP,config.WALLETPORT,"","")
    bot=MarketBalanceBot(wsapi,clientrpc)
    wsapi.connect()
    wsapi.connect().run_forever()

That means we don't have to make our own subclass of GrapheneWebsocketProtocol, instead make GrapheneWebsocketProtocol cannonical for all users, and have them just make classes that have the APIs as attributes of the class

Missing test for GenesisBalance

ValueError in amount.py

Version 1.1.11, while iterating on openorders:

  File "./get_balance.py", line 66, in <module>
    main()
  File "./get_balance.py", line 54, in main
    for order in account.openorders:
  File "/home/vvk/devel/golos/python-bitshares/bitshares/account.py", line 72, in openorders
    Order(o, blockchain_instance=self.blockchain) for o in self["limit_orders"]
  File "/home/vvk/devel/golos/python-bitshares/bitshares/account.py", line 72, in <listcomp>
    Order(o, blockchain_instance=self.blockchain) for o in self["limit_orders"]
  File "/home/vvk/devel/golos/python-bitshares/bitshares/price.py", line 152, in __init__
    blockchain_instance=self.blockchain,
  File "/home/vvk/.local/share/virtualenvs/market-scripts-BfRrkhQ4/lib/python3.6/site-packages/graphenecommon/instance.py", line 35, in __init__
    cls.__init__(self, *args, **kwargs)
  File "/home/vvk/.local/share/virtualenvs/market-scripts-BfRrkhQ4/lib/python3.6/site-packages/graphenecommon/amount.py", line 130, in __init__
    raise ValueError
ValueError

pricefeeds.py incorrect blockchain price displayed when quote is not BTS

Added some print statements to illustrate what happens. It looks like fetch_from_wallet expects the feed to have "BTS" as quote.

http://pastebin.com/hX7KEaGS

Consider transitioning away from assert for data validation

See Caveat #1 – Don’t Use Asserts for Data Validation

https://dbader.org/blog/python-assert-tutorial

Idea: split data structures from data loaders

Current classes (Asset, Amount, Price, etc) are responsible both for loading data via API and storing/handling this data (copy, multiplication, division, comparison etc)

The idea is to split data loading and data storing/processing to separate classes. One class is for loading object, and second to store object data and doing all data processing. This is needed to avoid loading of blockchain objects via API when instantiating new classes in various methods like __mul__, __div__ etc. This is important as it's almost impossible to make asyncio versions of magic methods. In new design magic methods are operate only on data storage classes without need to query API.

Here is the brief demonstration:

from asyncinit import asyncinit

class Data(dict):
    """ Actual data is stored here
    """
    def __init__(self, obj):
        # Do all usual things to load data into attributes etc
        self['foo'] = obj['foo']
        
    def copy():
        pass
    
    def __mul__():
        pass
    
    def __div__():
        pass
        
class Proxy(Data):
    """ Amount, Asset, Price, etc
    """
    def __init__(self, a):
        obj = self.load_data(a)
        self.__data = Data(obj)
    
    def load_data(self, o):
        """ RPC call here
        """
        return {'foo': 'bar'}
    
    def __getattr__(self, name):
        return getattr(self.__data, name)
    
@asyncinit
class ProxyAio(Proxy):
    """ Example async loader
    """
    async def __init__(self, a):
        obj = await self.load_data(a)
        self.__data = Data(obj)
    
    async def load_data(self, o):
        """ RPC call here
        """
        return {'foo': 'bar'}

Bitshares specifics in Blockchain.ops()

                    # Replace opid by op name
                    op[0] = self.operationids.getOperationNameForId(op[0])

This is bitshares-specific as another graphene chains may already have op name here.

ModuleNotFoundError: No module named 'grapheneapi.grapheneclient

Traceback (most recent call last):
File "Dbot5.py", line 12, in
from grapheneexchange import GrapheneExchange
File "/usr/local/lib/python3.6/dist-packages/graphenelib-0.4.2-py3.6.egg/grapheneexchange/init.py", line 1, in
File "/usr/local/lib/python3.6/dist-packages/graphenelib-0.4.2-py3.6.egg/grapheneexchange/exchange.py", line 1, in
ModuleNotFoundError: No module named 'grapheneapi.grapheneclient'

piston command not working

I am now getting error:

The 'websockets==2.0' distribution was not found and is required by steem

Infact it's already there

aconda3/anaconda/lib/python3.5/site-packages
Requirement already satisfied (use --upgrade to upgrade): asyncio in /anaconda3/anaconda/lib/python3.5/site-packages (from websockets==2.0)

Graphenelib 0.6.3 is broken

Just try to initialize bitshares instance and you will get:

% ./testcase.py 
Traceback (most recent call last):
  File "./testcase.py", line 7, in <module>
    bitshares = BitShares(node='wss://api.bts.blckchnd.com', no_broadcast=True)
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/bitshares-0.1.16-py3.6.egg/bitshares/bitshares.py", line 144, in __init__
    **kwargs)
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/bitshares-0.1.16-py3.6.egg/bitshares/bitshares.py", line 173, in connect
    self.rpc = BitSharesNodeRPC(node, rpcuser, rpcpassword, **kwargs)
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/bitshares-0.1.16-py3.6.egg/bitsharesapi/bitsharesnoderpc.py", line 24, in __init__
    self.chain_params = self.get_network()
AttributeError: can't set attribute

Fix JSON Decode Error

I've just tried follow() functionality in Piston which calls custom_json, which passes required_posting_auths, which is an Array of Strings, which causes json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) because it eventually tries to make JsonObj out of String

The fix is in #50

Blockchain.ops() wrong description

Yields all operations (including virtual operations) starting from

Should be "not including virtual operations" as it uses get_block() API call which only gives raw block content.

Aio: Unable to reconnect another node

I try to change node in connected aio.bitshares instance, but receive an error:

from bitshares.aio.graphene import BitShares

async def foo():
    bitshares_instance = BitShares(node="wss://node_1")
    await bitshares_instance.connect()
    assert bitshares_instance.is_connected()
    bitshares_instance.rpc.url = "wss://node_2"
    await bitshares_instance.connect()

---

/graphenecommon/aio/chain.py", line 43, in connect
    await self.rpc.connect()
/grapheneapi/aio/api.py", line 48, in connect
    await self.next()
grapheneapi/aio/api.py", line 56, in next
    await self.connection.disconnect()
AttributeError: 'function' object has no attribute 'disconnect'

trx.ref_block_prefix == tapos_block_summary.block_id._hash[1].value()

Despite the issue was addressed in b3d6cda, I started to get this this error time-to-time in my dexbot production instance.

Error means "referenced block num prefix != actual block prefix". Block prefix is taken from next block using "previous" field: block = ws.get_block_header(int(dynBCParams["last_irreversible_block_num"])+1)

Some ambiguity is caused by referencing block num using only 16 bits. This means, referenced block num should not be older than 65535 from head block. The only way I can reproduce this issue artificially is by referring to some old block, like this: dynBCParams["last_irreversible_block_num"] -= 70000

I'm not sure how it's happening in production, but I can guess there could be a reconnection to the next node and LIB on new node was shifted to the past (node out of sync). I added a debug to see LIB next time the issue will happen.

Deprecated cryptography method usage

In ecdsa.py signer method is used and it prints out a warning:

graphenelib-0.6.6-py3.5.egg/graphenebase/ecdsa.py:187: CryptographyDeprecationWarning: signer and verifier have been deprecated. Please use sign and verify instead.
  signer = private_key.signer(ec.ECDSA(hashes.SHA256()))

Missing test for Vesting

No test for Vesting class

watcher.py generates many local-wallet screens when wallet is large.

more of an info/wishlist than a bug.

Large wallets take longer than one second to load. The python script will keep trying until the wallet is loaded, sometimes spawning multiple screens. That's why it's a good idea to only have the needed keys, and cut away the cruft.

$ screen -ls
There are screens on:
15269.local-wallet (12/14/2015 10:22:40 AM) (Detached)
15263.local-wallet (12/14/2015 10:22:29 AM) (Detached)
15254.local-wallet (12/14/2015 10:22:18 AM) (Detached)
15248.local-wallet (12/14/2015 10:22:07 AM) (Detached)
14699.witness (12/14/2015 10:05:34 AM) (Detached)
14642.pts-0.bitshares2 (12/14/2015 10:04:18 AM) (Detached)

each local-wallet has a running cli_wallet. That causes problems with the pricefeed script as each cli_wallet is listening on the same port.

    while result == None:
        try:
            print("waiting ...")
            subprocess.call(["screen","-dmS","local-wallet",config.path_to_cli_wallet,"-H",config.rpc_port,"-w",config.path_to_wallet_json])
            time.sleep(1)
            result = rpc.info()
        except:
            time.sleep(10)
            pass

Either increase the timeout time.sleep(1), or only use watcher.py with smaller wallets.

Graphenelib 0.6.4 and 0.6.5 erroneously handling PrivateKey

#!/usr/bin/env python

#from bitsharesbase.account import PrivateKey
from graphenebase.account import PrivateKey

key = '6PRQdSDAjq41FVc58T1gykbihNPQeUCMWj8KW9uoeDKd1ZdZ6MbShYPhvS'

PrivateKey(key)
print('this should not happen')

Run testcase:

0.6.5:

this should not happen

0.6.3:

Traceback (most recent call last):
  File "./testcase2.py", line 8, in <module>
    PrivateKey(key)
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/graphenebase/account.py", line 322, in __init__
    self._pubkeyhex, self._pubkeyuncompressedhex = self.compressedpubkey()
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/graphenebase/account.py", line 331, in compressedpubkey
    order = ecdsa.SigningKey.from_string(secret, curve=ecdsa.SECP256k1).curve.generator.order()
  File "/home/vvk/.local/share/virtualenvs/DEXBot-XTl2tJdV/lib/python3.6/site-packages/ecdsa-0.13-py3.6.egg/ecdsa/keys.py", line 149, in from_string
    assert len(string) == curve.baselen, (len(string), curve.baselen)
AssertionError: (38, 32)

Suggestion for another price feed source

We could use https://api.uphold.com/v0/ticker