mlabs-haskell/bot-plutus-interface

Unofficial PAB implementation

bot-plutus-interface's Issues

Improve protocol params handling

These are still some vague ideas:

• Instead of having the protocol params strored in PABConfig, we should query the node for it (each time?)
• Generating a protocol params file if none exists (cardano-cli query protocol-parameters) (done by #91)

Testing and effect handling

using freer to handle effects, so we can test them

Saving transactions in unique files denominated by their txIds

To be able to handle multiple transaction in, we need to save each txbody (unsigned) and tx (signed) file with a unique name.

Fee calculation

Using a config file

I think our PABConfig got a bit bloated with a lot of options that don't necessarily need to be changed.
I would propose a yaml configuration file, but i am open to other options.

Balancing does not preserve output order

If there is no change UTxO, one is added to the start (it should be at the end)
Ada change handling always moves any existing change UTxO to the start of the outputs.

Better collateral handling

Problem

Currently, for each transaction we pick the first ada-only utxo we can find in the user's wallet, and use that as a collateral. There are two problems with this approach:

• a wallet might not have any utxos complying with this restriction (ada-only utxo)
• the utxo picked as collateral contains more ada than what the user would want to risk

Solution ideas

1. Always return at least one ada change

This would be the easier solution, with a change to balancing logic so that it always returns at least one ada-only change output.
#56

Problems:

• could increase dust
• min-utxo amount (roughly 1 ada) might not be enough to cover all collateral needs (TBH, this is not at all handled at the moment)

2. Separate utxo for collateral use only

This is what the nami wallet does, it separates 5 adas as collateral, and never uses that as regular input for transaction. We could do the same, by using a separate address for collaterals and regular inputs, or storing the collateral utxo in state and modifying queries so that it could never get used.

Problems:

• slightly more difficult to implement, than proposal 1.
• using two wallets would increase transaction fees

Rename project to something fake-pabish (to indicate that this is not a full-fledged PAB)

Add inline Datum support for `utxosAt` node query.

Target branch: gergely/vasil

Current cardano-cli call does not support inline Datums.

Task is to add support for TxOuts with inline Datums.

Solution can be implemented as partial solution of #109 and #101 applied only to utxosAt node query.

Budget estimation wrong

Even using the IOHK provided functionality, budgets are wrong.
Adding a hardcode budget feature to solve this in the short term.

Transaction confirmation

Waiting one block for a transaction is not enough.
We ideally need to implement the AwaitTxOutStatusChangeReq request

Occasional chain index timeout

Every now and then when running a contract, chain index times out and errors, we should increase the timeout for this

Use TypeApplications instad of Proxy for the main endpoint

Change this:

MLabsPAB.runPAB pabConf (Proxy :: Proxy MyContract)

to this:

MLabsPAB.runPAB @MyContract pabConf

Chain-index pagination

Multisig support

This might require the cardano-wallet integration first

Query local node to convert between `POSIXTime` and `SlotNo`

Currently bot-plutus-interface uses plutus-ledger to convert between POSIXTime and SlotNo, however that does not necessarily match what the cardano node is doing, because slot length can change along the lifetime of the chain (and has changed on testnet).

Inaccurate conversion affects in particular validators that try to match the validity range of a transaction with info stored in the datums.

I documented in this comment Plutonomicon/cardano-transaction-lib#410 (comment) how cardano-ledger translates the SlotNo in the validity range during validation.

Ultimately it relies on the slotToWallclock query from ouroboros-network (which is used to build the EpochInfo record).
This latter package also has a wallclockToSlot query to convert the other way:
https://github.com/input-output-hk/ouroboros-network/blob/bd9e5653647c3489567e02789b0ec5b75c726db2/ouroboros-consensus/src/Ouroboros/Consensus/HardFork/History/Qry.hs#L461-L481

I believe that BPI should base all time and slot conversion on such queries (or maybe EpochInfo for the slot to time direction).
Most operations where we need this already query the node for some info anyway so I think the extra EraHistory query necessary should not cause responsiveness problems.

Endpoint changes, build examples

Change the TxId param to raw-tx to use its own capture, and validate correctly.
Also - the examples weren't updated, and aren't built as part of CI
We should update them to build, and maybe add them to CI.

Introduce QueryNode effect for interactions with node

There is no explicit logic expressed in terms of effect for interactions with node, see #105 (comment)

Introducing it will lead to cleaner architecture, easier testing (see #105 (comment)) and will benefit #101

Contract submission error in vasil

I want to submit contract in bpi vasil v2 and getting error:

[BPI] [DEBUG] [BPI] [DEBUG] Balance tx: BalanceTxFailed: Insufficient funds: ScriptFailure (ScriptErrorMissingScript (RdmrPtr Spend 0) (fromList [(RdmrPtr Spend 0,(Spending (TxIn (TxId {_unTxId = SafeHash "8ff883d61d819c051a9ad0e4005987f23bf25cfe54f3b2e8c6a7d260e899cd0a"}) (TxIx 0)),Nothing,ScriptHash "6225590d50a2ef5f6226e2863a263ec005d2a8f07b660f966f3765a9"))]))
after i started debug balanceTxIO in bpi line by line and don't understand:

bot-plutus-interface/src/BotPlutusInterface/CardanoCLI.hs

Line 163 in 56e18fb

callCommand @w $ ShellArgs "cardano-cli" (opts ins mints) (const $ valBudget <> mintBudget)

after this line result is
res <- callCommand is ExBudget {exBudgetCPU = ExCPU 0, exBudgetMemory = ExMemory 0} and transaction is build and written on disk.
But after line

bot-plutus-interface/src/BotPlutusInterface/BodyBuilder.hs

Line 41 in 56e18fb

>>= buildBodyUsingEstimatedBudget

, where it should return previous result it drops error with ScriptErrorMissingScript

Add port to config

Port is hard coded to 9080. I'd like a way to run multiple PABs at once on different ports, so ideally port becomes part of the config.

Remove temp `tx` files from `txs` directory?

When balancing a transaction, we create many transaction files until settling on the correct one. While it makes sense to keep the final file for logs/accountibility, the temporary transaction should probably be deleted.
Open to discussion here

`calculateExBudget` missing ScriptContext

The calculateExBudget doesn't provide a script context to the scripts it attempts to predict for (and thus has always been broken). As such, the scripts dont actually get run, and the budget we get back is simply the cost of argument application.
We'll need to build a dummy script context out of the various info we know about the tx, and use that.
This issue was discovered follow my change to fully balance, at which point, we stopped relying on cardano-cli's internal budgetting, and instead used our own broken one. This causes any transaction involving a script to fail.

This issue is the direct cause of #72

Get a staging branch

Set up a staging branch with protections

NFT example fails

"The budget was overspent" error on testnet

Coin selection algorithm

We don't have any sophisticated coin selection working at the moment, the order in which pre balance pick utxos for the inputs is based on the order they appear on the cli. This could cause problems, when there are a lot of dust utxos, or the token we are looking for is too much down the list. In worst cases, this could bloat the tx size, fees, or even fail submitting.

Should Ada change get its own UTxO?

Currently, in say a transaction where we send a token from one wallet to another, where the sender has a minimum ada UTxO with the special token, and a second UTxO with ada only (which is used for collateral), the change for that transaction (so the special tokens AND the excess Ada) end up in one UTxO.
This means that now we have no ada-only collaterals in the wallet - it essentially bricks itself until it receives Ada externally.
Perhaps all change Ada should always end up in a separate UTxO - if the change is higher than the min UTxO amount.

Curious of peoples thoughts here.

Update docs

Upgrade to latest plutus version using flakes

Remote has bugs

The ensure folder exists function creates a folder on the local machine, then doesn't have it on the remote, and fails.
This causes issues with the txs folder.

Also, signing-keys has some odd behavior, needs some research.

Catch CLI errors properly

Currently errors from the cardano-cli throw an IOErr, these are not caught properly, we should try to use readProcessWithExitCode for a nicer behavior.

Build examples on the CI

I don't know what would be the good way to build examples on the CI, but leaving them isolated from the library. I would appreciate if someone could give me advice on this, or direct me to some other library with the same kind of architecture.

cardano-wallet integration

I started this integration task, but it is incomplete at the moment. These are the necessary tasks:

• Solve wallet-key encoding issue: walletId used by the cardano-wallet api is hash of the walledId that the plutus library uses. I implemented this conversion in the current version, but unfortunately it is incorrect, and I couldn't find out the reason for the problem.
• Coin selection: replacing the current collectTxIns function
• Balancing transactions with wallet: this feature of the wallet is still in development
• Signing by the wallet

Get raw tx endpoint

While creating transactions, this projects write them to a txs folder (by default, the actual directory is in config) (either locally or remote). They go through 3 stages - first the raw file is created, this is a balanced unsigned transaction, then the signed file is created, simply signing the raw, then that file is submitted to the chain.

We need a servant endpoint that will allow us to read the raw file back to a http client.
Raw transaction files are named by tx-[theirhash].raw, so this endpoint will take theirhash, as simply a Text, lookup the transaction that matches, and return it as JSON (the raw file is a json object with a cborHex field containing most of the actual transaction).
It should ensure that the hash passed in does not try to move to different directories (so, resolve path and ensure its in ./txs/), and should never be allowed to read a signed file.

This endpoint also may only work if a config flag is set, something like enableTxEndpoint or whatever seems most relevant. This should default to False.

Update token name handling for the cli

As stated in the release not of cardano-cli 1.32, token names are expected to be in a hex format instead of ascii.

Handle new PAB requests (Vasil update branch)

The latest branches of plutus-apps introduces new effects that we have to handle:
https://github.com/input-output-hk/plutus-apps/blob/next-node/plutus-contract/src/Plutus/Contract/Effects.hs#L115-L134

Should be added here, and while we're at it, we should remove the catch all case, so we can better track API changes:
https://github.com/mlabs-haskell/bot-plutus-interface/blob/gergely/vasil/src/BotPlutusInterface/Contract.hs#L201

base branch should be https://github.com/mlabs-haskell/bot-plutus-interface/tree/gergely/vasil

Misleading error when minting value is 0

When user tries to submit transaction that mints 0 of particular token returned error is InsufficientFunds:

Contract failed with ContractExecutionError
          "WalletError (InsufficientFunds \"ExitCode 1: Missing: --mint VALUE\\n\\nUsage: cardano-cli transaction build-raw \\n            [ --byron-era\\n            | --shelley-era\\n            | --allegra-era\\n            | --mary-era\\n            | --alonzo-era\\n            ]\\n            [--script-valid | --script-invalid]\\n            (--tx-in TX-IN\\n              [--tx-in-script-file FILE\\n                [(--tx-in-datum-file FILE | --tx-in-datum-value JSON VALUE)\\n                  ( --tx-in-redeemer-file FILE\\n                  | --tx-in-redeemer-value JSON VALUE\\n                  )\\n                  --tx-in-execution-units (INT, INT)]])\\n            [--tx-in-collateral TX-IN]\\n            [--required-signer FILE | --required-signer-hash HASH]\\n            [--tx-out ADDRESS VALUE\\n              [ --tx-out-datum-hash HASH\\n              | --tx-out-datum-hash-file FILE\\n              | --tx-out-datum-hash-value JSON VALUE\\n              | --tx-out-datum-embed-file FILE\\n              | --tx-out-datum-embed-value JSON VALUE\\n              ]]\\n            [--mint VALUE\\n              (--mint-script-file FILE\\n                [(--mint-redeemer-file FILE | --mint-redeemer-value JSON VALUE)\\n                  --mint-execution-units (INT, INT)])]\\n            [--invalid-before SLOT]\\n            [--invalid-hereafter SLOT]\\n            [--fee LOVELACE]\\n            [--certificate-file CERTIFICATEFILE\\n              [--certificate-script-file FILE\\n                [\\n                  ( --certificate-redeemer-file FILE\\n                  | --certificate-redeemer-value JSON VALUE\\n                  )\\n                  --certificate-execution-units (INT, INT)]]]\\n            [--withdrawal WITHDRAWAL\\n              [--withdrawal-script-file FILE\\n                [\\n                  ( --withdrawal-redeemer-file FILE\\n                  | --withdrawal-redeemer-value JSON VALUE\\n                  )\\n                  --withdrawal-execution-units (INT, INT)]]]\\n            [--json-metadata-no-schema | --json-metadata-detailed-schema]\\n            [--auxiliary-script-file FILE]\\n            [--metadata-json-file FILE | --metadata-cbor-file FILE]\\n            [--genesis FILE | --protocol-params-file FILE]\\n            [--update-proposal-file FILE]\\n            --out-file FILE\\n\\n  Build a transaction (low-level, inconvenient)\\n\\n  \\ESC[93mPlease note the order of some cmd options is crucial. If used incorrectly may produce undesired tx body. See nested [] notation above for details.\\ESC[0;22;23;24m\\n\")"

Switch CI to Hydra

See #devops pinned messages for how.

Expand mock filesystem

The current mock file system only accounts for files with arbitrary file paths, in one location.
Ideally, this should expand to handle directory checks (ensuring one cannot create a file in a directory that was not previously created), as well as location - when running in remote mode, we have the local file system and the remote file system, both of which need to act correctly.

Use cardano-api instead of cli calls

This is a considerably big change, but we could gain better performance and simpler setup with it.

Upgrade plutus, incorporate full addressees with `MustPayToPubKeyAddress` correctly.

Addresses change based on if their staking address is present, and the plutus versions we're on now only allows specifying a pubkeyhash, meaning tokens sent via this don't end up at the expected address hash. If we upgrade plutus, we get this functionality, and can forward it to the cli to use the full addresses

Update dependencies

This is quite far behind, including the index-state.

Plutus V2

The Plutus V2 upgrade is going through the following steps:

• on-chain Vasil compliant version (without contract execution)
• off-chain Vasil compliant version (with contract execution, but without V2 off-chain features)
• off-chain Vasil compatible version (with all the whistles and bells)

Wait length configurability

For wait slots functionality we use recurring call to cardano-cli to get the tip of the chain. The wait time between these calls is hard coded, which is too long for some use cases (for example testing on local cluster)

Direct connection

Currently, we mostly use this project to deploy a pab for a bot, and have the bot only call said pab on the same machine.
I propose some way of having a haskell bot interact with the project directly, rather than risking http access externally.
Bots usually just need a way to run a contract, and see state changes. We could model this as some sort of runContract :: PABConfig -> Contract w s e a -> (StateChange -> IO ()) -> IO () passing a state handler in.

@gege251 Thoughts?

Threat of plutus-apps getting obsolete

There's a possibility that plutus-apps (PAB) will not be supported in the future, so we might need to maintain it ourselves, or move away from it altogether.
This could potentially be a good moment to refactor some of our code, once we don't need to conform to the interface of plutus-apps.

Split CLI and budget calculation into separate modules

They shouldn't really be together

Transaction change should not be added to UTxOs with Datums

This breaks some logic for dapps (namely seabug)

Change output added by the CLI may have a too little ada value

When the cli builds the transaction, it adds the change output manually. However, in case the input side value is just enough to cover the outputs and fees, the change output might have too little lovelaces remaining, and the transaction fails with Min utxo amount is not reached error.
It could be an easy fix, as we already know the fees and the min utxo for an ADA only change output, and having a surplus of fees + minUtxo amount should suffice.
Or we could even balance the whole transaction manually at that point... this is up for debate.

Update documentation

awaitTxStatusChange hides rollbacks as `Unknown`

awaitTxStatusChange currently only returns Unknown or Committed.

In case of a rollback then we only ever get Unknown, which e.g. makes awaitTxConfirmed keep looping as if we were still waiting for the tx to show up at all.

As we discussed before, we should have awaitTxStatusChange actually return the current status of the transaction, rather than include the 8 block wait for confirmation.

Utilities built on top of this primitive can then implement an appropriate way to deal with rollbacks when they see TentativelyConfirmed followed by Unknown