TransactionBuilder doesn't work if there's no index 0 utxo input

[python-cardano]

TransactionBuilder doesn't work if there's no index 0 utxo input

[34r7h]

Ok you're right. I was testing with addresses having less than ~2.4 ada so was failing as expected. I'll close this issue as it's really the same problem you've fixed in a recent commit. Will test and follow-up if same trouble happens later. Thanks for everything!

[cffls]

Thanks for reporting the issue! It seems to be a bug in UTxO selector. Will look into this.

[cffls]

Hi @34r7h , the bug should be fixed under this commit: cffls@e00b569 Please checkout the latest code and let me know if it works. Thanks!

[34r7h]

Hey @cffls, thanks for your swiftness.

I think you solved an unrelated issue.. this bug seems to come when the coin selection doesn't have a 0 indexed input to choose from.

I checked that the version is 0.4.1. Notice the first transaction log has 2 utxos available:

utxos [{'input': {'index': 1,
 'transaction_id': TransactionId(hex='9d255cdacd8a575ee86f4ad0a61b14c7be037c623059f71b1bc9ce8d4e53cb6c')},
 'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
 'amount': 2821804,
 'datum_hash': None}}, {'input': {'index': 0,
 'transaction_id': TransactionId(hex='9c9c7ce1ed9018b31c8a3e94475e7f607d87be914091e40d6b44efc711146811')},
 'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
 'amount': 1000000,
 'datum_hash': None}}]

While the following transaction has only one utxo (with index: 1) fails:

utxos [{'input': {'index': 1,
 'transaction_id': TransactionId(hex='81a471965bfdaf4bc570f7a987c9dbe1bbf01a08564de4c56319746394b5839e')},
 'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
 'amount': 2652255,
 'datum_hash': None}}]
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/txbuilder.py", line 658, in build
    selected, _ = selector.select(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/coinselection.py", line 109, in select
    additional, _ = self.select(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/coinselection.py", line 94, in select
    raise InsufficientUTxOBalanceException("UTxO Balance insufficient!")
pycardano.exception.InsufficientUTxOBalanceException: UTxO Balance insufficient!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/347rh/a/cardano-python-js/python/createtx.py", line 39, in <module>
    signed_tx = builder.build_and_sign([sk], change_address=address)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/txbuilder.py", line 767, in build_and_sign
    tx_body = self.build(change_address=change_address)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/txbuilder.py", line 690, in build
    raise UTxOSelectionException(
pycardano.exception.UTxOSelectionException: All UTxO selectors failed.
Requested output:
 {'coin': 1158901, 'multi_asset': {}} 
Pre-selected inputs:
 {'coin': 0, 'multi_asset': {}} 
Additional UTxO pool:
 [{'input': {'index': 1,
 'transaction_id': TransactionId(hex='81a471965bfdaf4bc570f7a987c9dbe1bbf01a08564de4c56319746394b5839e')},
 'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
 'amount': 2652255,
 'datum_hash': None}}] 
Unfulfilled amount:
 {'coin': -1493354, 'multi_asset': {}}

[34r7h]

As a simple hotfix, I'm checking if there's only one utxo and adding it manually. The rest of your logic seems happily intact to complete the tx with proper plumbing for change, etc. <3

if len(utxos) == 1:
    print('Only one UTxO. adding raw input..')
    builder.add_input(utxos[0])

[cffls]

HI @34r7h , I couldn't reproduce the issue after the fix I mentioned above. Here is the code I am running:

network = Network.MAINNET
context = BlockFrostChainContext("mainnetqEZ4wDDoRdtWqh2SNVLNqfQbhlNmTbza", network)

address = Address.from_primitive('addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d')

builder = TransactionBuilder(context)
builder.add_input_address(address)
utxos = context.utxos(str(address))

builder.add_output(
    TransactionOutput(
        Address.from_primitive(
"addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee"
        ),
        Value.from_primitive(
            [
                1000000,
            ]
        ),
    )
)
signed_tx = builder.build(change_address=address)

It finished successfully without exception. The only difference was that I used build instead of build_and_sign, but the underlying selection logic should be exactly the same. Could you please try again with the latest code of pycardano? Thank you!

[34r7h]

Hey Jer sorry for the delay. I'm still getting the error but using the conditional workaround:

if len(utxos) == 1:
    print('add raw input')
    builder.add_input(utxos[0])

When you tried to reproduce, were you successful with an address containing only 1 utxo?

[cffls]

Hi @34r7h , the address addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d has two utxos, so I modified the code to make sure the chain context only return one, pasted below. You can see that the builder was not called with add_input but still able to build the transaction with only 1 utxo input.

from dataclasses import dataclass, field
from typing import Dict, List

from pycardano import *
network = Network.MAINNET
context = BlockFrostChainContext("mainnetqEZ4wDDoRdtWqh2SNVLNqfQbhlNmTbza", network)

address = Address.from_primitive('addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d')

builder = TransactionBuilder(context)
builder.add_input_address(address)
utxos = context.utxos(str(address))

# Force chain context to return only one utxo
context.utxos = lambda _: utxos[-1:]

builder.add_output(
    TransactionOutput(
        Address.from_primitive(
"addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee"
        ),
        Value.from_primitive(
            [
                1000000,
            ]
        ),
    )
)
tx = builder.build(change_address=address)

print(tx.inputs)

print(tx.outputs)

Output:

[{'index': 1,
 'transaction_id': TransactionId(hex='e6e9ab73f95939c04be8f4f07af9eac7028a12a9f6b1fc2f5dc19509f543da23')}]
[{'address': addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee,
 'amount': {'coin': 1000000, 'multi_asset': {}},
 'datum_hash': None}, {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
 'amount': {'coin': 37118232, 'multi_asset': {}},
 'datum_hash': None}]

[34r7h]

Hi Jerry, not quite the issue brother.. this bug happens if and only if there's one transaction to choose from. The address above has multiple txs already so we can't reproduce.

In the case of a fresh addr, the context.utxos = lambda _: utxos[-1:] results in

 raise UTxOSelectionException(
0|crypto |     pycardano.exception.UTxOSelectionException: All UTxO selectors failed.
0|crypto |     Requested output:
0|crypto |      {'coin': 1189528, 'multi_asset': {}} 
0|crypto |     Pre-selected inputs:
0|crypto |      {'coin': 0, 'multi_asset': {}} 
0|crypto |     Additional UTxO pool:
0|crypto |      [] 

So for some reason, the builder isn't selecting the single transaction as a valid utxo. I'm still using builder.add_input(utxos[0]) if len(utxos) == 1 and that's working fine in this case. I suspect the context return from BF is causing this

Can you try to reproduce again with a fresh address or lmk if there's a mistake I'm missing in this code?

from pycardano import *
import json
import sys
from dataclasses import dataclass, field
from typing import Dict, List

args = sys.argv[1:]
secret = args[0]
data = args[1]
bf = args[2]
dev = False
jsonsecret = json.loads(secret)
jsondata = json.loads(data)

pkey = jsonsecret["payment"]["signing"]["cborHex"]
vkey = jsonsecret["payment"]["verification"]["cborHex"]

network = Network.MAINNET
context = BlockFrostChainContext(bf, network)

sk = PaymentSigningKey.from_cbor(pkey)
vk = PaymentVerificationKey.from_signing_key(sk)
address = Address.from_primitive(jsondata["address"])

builder = TransactionBuilder(context)
utxos = context.utxos(str(address))

if len(utxos) == 1:
    builder.add_input(utxos[0])
    # context.utxos = lambda _: utxos[-1:]
else:
    builder.add_input_address(address)

for x in jsondata["outputs"]:
    outputaddress = x["address"]
    tokens = [2000000]

    for y in x["tokens"]:
        if y["unit"] == "lovelace":
            tokens[0] = int(y["quantity"])
        else:
            policyid = y["unit"][0 : 56]
            tokenname = y["unit"][-30:len(y["unit"])]
            tokens.append(
                {
                    bytes.fromhex(policyid): {
                        bytes.fromhex(tokenname): int(y["quantity"])  # Asset name and amount
                    }
                }
            )
    builder.add_output(
        TransactionOutput(
            Address.from_primitive(outputaddress), Value.from_primitive(tokens)
        )
    )

signed_tx = builder.build_and_sign([sk], change_address=address)
tx_id = str(signed_tx.id)
context.submit_tx(signed_tx.to_cbor())

[cffls]

Hi @34r7h , I didn't mean to let you to put context.utxos = lambda _: utxos[-1:] in your production code. I was using it simply to demonstrate that the bug has been fixed. Because the example I posted contains more than one utxo, I had to fake its utxos so the build will only see one in the list.

Previously, I meant your code will still work fine with this simplification (please make sure the version of pycardano has been upgraded to v0.5.0):

changing this:

if len(utxos) == 1:
    builder.add_input(utxos[0])
    # context.utxos = lambda _: utxos[-1:]
else:
    builder.add_input_address(address)

to this:

builder.add_input_address(address)

If you have an address that contains only one utxo, I am happy to test it out for you.

[34r7h]

Ah, ye this issue is when only one tx exists on an address. Here's an address with only one:
addr1qypm6f2z5g45duzj9v9lt7jz9ce2q5m59vw3reqm9e25uxpynes82004nuvufjx0zu8up9dlr574azfnnp2vj3dcwrsqfux5t0

[cffls]

It is working correctly with the address you provided:

network = Network.MAINNET
context = BlockFrostChainContext("mainnetqEZ4wDDoRdtWqh2SNVLNqfQbhlNmTbza", network)

address = Address.from_primitive('addr1qypm6f2z5g45duzj9v9lt7jz9ce2q5m59vw3reqm9e25uxpynes82004nuvufjx0zu8up9dlr574azfnnp2vj3dcwrsqfux5t0')

builder = TransactionBuilder(context)
builder.add_input_address(address)

builder.add_output(
    TransactionOutput(
        Address.from_primitive(
"addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee"
        ),
        Value.from_primitive(
            [
                1000000,
            ]
        ),
    )
)
tx_body = builder.build(change_address=address)
print(tx_body)

Output:

{'auxiliary_data_hash': None,
 'certificates': None,
 'collateral': None,
 'collateral_return': None,
 'fee': 167965,
 'inputs': [{'index': 0,
 'transaction_id': TransactionId(hex='1764ea2ce4653c1f03b78a5ac73cf4c40247a930f60d3467927f744e9c06fc6d')}],
 'mint': None,
 'network_id': None,
 'outputs': [{'address': addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee,
 'amount': {'coin': 1000000, 'multi_asset': {}},
 'datum_hash': None},
             {'address': addr1qypm6f2z5g45duzj9v9lt7jz9ce2q5m59vw3reqm9e25uxpynes82004nuvufjx0zu8up9dlr574azfnnp2vj3dcwrsqfux5t0,
 'amount': {'coin': 1662574, 'multi_asset': {}},
 'datum_hash': None}],
 'reference_inputs': None,
 'required_signers': None,
 'script_data_hash': None,
 'total_collateral': None,
 'ttl': None,
 'update': None,
 'validity_start': None,
 'withdraws': None}

[34r7h]

Cheers for continuing to humor me on this.. yes, build works for me too but build_and_sign throws the error. Could there be a hiccup in the extra fee for the sig?

I'm fine with my conditional check, so nothing is breaking and all good. At your convenience, plz check build_and_sign from an address you can sign. To reproduce, it must have only one tx

[cffls]

Hi @34r7h , I find it hard to believe build works but build_and_sign throws error. What error did you see?

I created an address that has only one UTxO in it and ran the following code. Everything is working correctly.

Code:

from blockfrost import BlockFrostApi, ApiUrls

from pycardano import *

network = Network.TESTNET

PAYMENT_KEY_PATH = "payment2.skey"

context = BlockFrostChainContext("my_testnet_project_id", network)

psk = PaymentSigningKey.load(PAYMENT_KEY_PATH)
pvk = PaymentVerificationKey.from_signing_key(psk)

address = Address(pvk.hash(), network=network)

print("Address: ", address)
print("UTxOs: ", context.utxos(str(address)))

builder = TransactionBuilder(context)
builder.add_input_address(address)

builder.add_output(
    TransactionOutput(
        Address.from_primitive(
            "addr_test1vzvj0223pmnnyyjkcqwnpt0rszlk5rtpdp3d7necq60wzmgt4h5rr"
        ),
        Value.from_primitive(
            [
                1000000,
            ]
        ),
    )
)

tx = builder.build_and_sign(change_address=address, signing_keys=[psk])

print("Transaction: ", tx)

Output:

Address:  addr_test1vqdfs0vy0eraw5xcpj89qkp0v38v2g6xkzzajp33tzs04vq8hq40k
UTxOs:  [{'input': {'index': 0,
 'transaction_id': TransactionId(hex='e22ea2413a8416b8169a3c8a5180c9a54cd8a463eaaf49a527812e4bdd7bcc8e')},
 'output': {'address': addr_test1vqdfs0vy0eraw5xcpj89qkp0v38v2g6xkzzajp33tzs04vq8hq40k,
 'amount': {'coin': 2800000, 'multi_asset': {}},
 'datum_hash': None}}]
Transaction:  {'auxiliary_data': None,
 'transaction_body': {'auxiliary_data_hash': None,
 'certificates': None,
 'collateral': None,
 'collateral_return': None,
 'fee': 165413,
 'inputs': [{'index': 0,
 'transaction_id': TransactionId(hex='e22ea2413a8416b8169a3c8a5180c9a54cd8a463eaaf49a527812e4bdd7bcc8e')}],
 'mint': None,
 'network_id': None,
 'outputs': [{'address': addr_test1vzvj0223pmnnyyjkcqwnpt0rszlk5rtpdp3d7necq60wzmgt4h5rr,
 'amount': {'coin': 1000000, 'multi_asset': {}},
 'datum_hash': None},
             {'address': addr_test1vqdfs0vy0eraw5xcpj89qkp0v38v2g6xkzzajp33tzs04vq8hq40k,
 'amount': {'coin': 1634587, 'multi_asset': {}},
 'datum_hash': None}],
 'reference_inputs': None,
 'required_signers': None,
 'script_data_hash': None,
 'total_collateral': None,
 'ttl': None,
 'update': None,
 'validity_start': None,
 'withdraws': None},
 'transaction_witness_set': {'bootstrap_witness': None,
 'native_scripts': None,
 'plutus_data': None,
 'plutus_script': None,
 'redeemer': None,
 'vkey_witnesses': [{'signature': b"\x0f\xb035[\xfc;\xa2\xca\xedbk\x84';h\xfa\xf0=\x92\xc5?\xff\xdb"
              b'\x06J\x12\x0b\x86\x80\xc7\x07Tk\xad\xf1\xe9\xf6\xb7?'
              b'\xebG\xfb\xec\r\xf9w\x9bg\xa3\xfc\xc8d8\x80#B\x83Q\xd5'
              b'g\xa4\xb6\x00',
 'vkey': {"type": "PaymentVerificationKeyShelley_ed25519", "description": "PaymentVerificationKeyShelley_ed25519", "cborHex": "58209ce6f79cc94658844a8651607242b6e02388da183dcecfd62389aad676597ac1"}}]},
 'valid': True}