komodoplatform/komodo

Komodo

---

Komodo

This is the official Komodo sourcecode repository based on https://github.com/jl777/komodo.

Development Resources

• Komodo Website: https://komodoplatform.com
• Komodo Blockexplorer: https://kmdexplorer.io
• Komodo Discord: https://komodoplatform.com/discord
• Forum: https://forum.komodoplatform.com
• Mail: [email protected]
• Support: https://support.komodoplatform.com/support/home
• Knowledgebase & How-to: https://support.komodoplatform.com/en/support/solutions
• API references & Dev Documentation: https://developers.komodoplatform.com
• Blog: https://blog.komodoplatform.com
• Whitepaper: Komodo Whitepaper
• Komodo Platform public material: Komodo Platform public material

List of Komodo Platform Technologies

• Delayed Proof of Work (dPoW) - Additional security layer and Komodos own consensus algorithm
• zk-SNARKs - Komodo Platform's privacy technology for shielded transactions
• Tokens/Assets Technology - create "colored coins" on the Komodo Platform and use them as a layer for securites
• Reward API - Komodo CC technology for securities
• CC - Crypto Conditions to realize "smart contract" logic on top of the Komodo Platform
• Jumblr - Decentralized tumbler for KMD and other cryptocurrencies
• Assetchains - Create your own Blockchain that inherits all Komodo Platform functionalities and blockchain interoperability
• Pegged Assets - Chains that maintain a peg to fiat currencies
• Peerchains - Scalability solution where sibling chains form a network of blockchains
• More in depth covered here
• Also note you receive 5% Active User Reward on your balance.
  See this article for more details

Tech Specification

• Max Supply: 200 million KMD
• Block Time: 60 seconds
• Block Reward: 3 KMD
• Mining Algorithm: Equihash

About this Project

Komodo is based on Zcash and has been extended by our innovative consensus algorithm called dPoW which utilizes Bitcoin's hashrate to store Komodo blockchain information into the Bitcoin blockchain. Other new and native Komodo features are the privacy technology called JUMBLR, our assetchain capabilities (one click plug and play blockchain solutions) and a set of financial decentralization and interoperability technologies. More details are available under https://komodoplatform.com/ and https://blog.komodoplatform.com.

Getting started

Dependencies (Ubuntu 20.04)

#The following packages are needed:
sudo apt-get install build-essential pkg-config libc6-dev m4 g++-multilib autoconf libtool ncurses-dev unzip git python3 python3-zmq zlib1g-dev wget libcurl4-gnutls-dev bsdmainutils automake curl libsodium-dev

Build Komodo

This software is based on zcash and considered experimental and is continously undergoing heavy development.

The dev branch is considered the bleeding edge codebase while the master-branch is considered tested (unit tests, runtime tests, functionality). At no point of time do the Komodo Platform developers take any responsbility for any damage out of the usage of this software. Komodo builds for all operating systems out of the same codebase. Follow the OS specific instructions from below.

Linux

git clone https://github.com/komodoplatform/komodo --branch master --single-branch
cd komodo
./zcutil/fetch-params.sh
./zcutil/build.sh -j$(expr $(nproc) - 1)
#This can take some time.

OSX

Ensure you have brew and Command Line Tools installed.

# Install brew
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
# Install Xcode, opens a pop-up window to install CLT without installing the entire Xcode package
xcode-select --install 
# Update brew and install dependencies
brew update
brew upgrade
brew tap discoteq/discoteq; brew install flock
brew install autoconf autogen automake
brew update && brew install gcc@8
brew install binutils
brew install protobuf
brew install coreutils
brew install wget
# Clone the Komodo repo
git clone https://github.com/komodoplatform/komodo --branch master --single-branch
# Change master branch to other branch you wish to compile
cd komodo
./zcutil/fetch-params.sh
./zcutil/build-mac.sh -j$(expr $(sysctl -n hw.ncpu) - 1)
# This can take some time.

Windows

Use a debian cross-compilation setup with mingw for windows and run:

sudo apt-get install build-essential pkg-config libc6-dev m4 g++-multilib autoconf libtool ncurses-dev unzip git python python-zmq zlib1g-dev wget libcurl4-gnutls-dev bsdmainutils automake curl cmake mingw-w64 libsodium-dev libevent-dev
curl https://sh.rustup.rs -sSf | sh
source $HOME/.cargo/env
rustup target add x86_64-pc-windows-gnu

sudo update-alternatives --config x86_64-w64-mingw32-gcc
# (configure to use POSIX variant)
sudo update-alternatives --config x86_64-w64-mingw32-g++
# (configure to use POSIX variant)

git clone https://github.com/jl777/komodo --branch master --single-branch
cd komodo
./zcutil/fetch-params.sh
./zcutil/build-win.sh -j$(expr $(nproc) - 1)
#This can take some time.

komodo is experimental and a work-in-progress. Use at your own risk.

To reset the Komodo blockchain change into the ~/.komodo data directory and delete the corresponding files by running rm -rf blocks chainstate debug.log komodostate db.log

Create komodo.conf

Create a komodo.conf file:

mkdir ~/.komodo
cd ~/.komodo
touch komodo.conf

#Add the following lines to the komodo.conf file:
rpcuser=yourrpcusername
rpcpassword=yoursecurerpcpassword
rpcbind=127.0.0.1
txindex=1
addnode=77.75.121.138
addnode=95.213.238.100
addnode=94.130.148.142
addnode=103.6.12.105
addnode=139.99.209.214
addnode=185.130.212.13
addnode=5.9.142.219
addnode=200.25.4.38
addnode=139.99.136.148

Create your own Blockchain based on Komodo

Komodo allows anyone to create a runtime fork which represents an independent Blockchain. Below are the detailed instructions: Setup two independent servers with at least 1 server having a static IP and build komodod on those servers.

On server 1 (with static IP) run:

./komodod -ac_name=name_of_your_chain -ac_supply=100000 -bind=ip_of_server_1 &

On server 2 run:

./komodod -ac_name=name_of_your_chain -ac_supply=100000 -addnode=ip_of_server_1 -gen &

Komodo is based on Zcash which is unfinished and highly experimental. Use at your own risk.

License

For license information see the file COPYING.

NOTE TO EXCHANGES: https://bitcointalk.org/index.php?topic=1605144.msg17732151#msg17732151 There is a small chance that an outbound transaction will give an error due to mismatched values in wallet calculations. There is a -exchange option that you can run komodod with, but make sure to have the entire transaction history under the same -exchange mode. Otherwise you will get wallet conflicts.

To change modes:

a) backup all privkeys (launch komodod with -exportdir=<path> and dumpwallet)
b) start a totally new sync including wallet.dat, launch with same exportdir
c) stop it before it gets too far and import all the privkeys from a) using komodo-cli importwallet filename
d) resume sync till it gets to chaintip

For example:

./komodod -exportdir=/tmp &
./komodo-cli dumpwallet example
./komodo-cli stop
mv ~/.komodo ~/.komodo.old && mkdir ~/.komodo && cp ~/.komodo.old/komodo.conf ~/.komodo.old/peers.dat ~/.komodo
./komodod -exchange -exportdir=/tmp &
./komodo-cli importwallet /tmp/example

---

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

komodo's Issues

Request for new parameter

Is this parameter available to use in smart chain ? I would like this parameter to implemented in the smart chain i have created.

Originally posted by @rusaik in #410 (comment)

MacOS builds failing in GHA

Refactor komodo_state struct

The komodo_state struct is used in many areas of Komodo. Refactoring 2 areas will improve reliability, memory use, and code readability:

1. NPOINTS - This is implemented as an array
   • A different collection from the standard library will make adding and removing elements easier to understand
   • The collection is often searched linearly. last_NPOINTSi tries to help. But a std::map or something similar may bring a performance improvement.
2. There is an array of komodo_state structs, indexed by CURRENCY, with some tricks around the chain token (always index 0) and KMD (always index 34). It seems CURRENCY actually has little involvement, and the link between komodo_state and CURRENCY should probably be broken. This may be a good use for a std::list

Potential Deadlock detected with cs_vSend cs_vRecvMsg

Received couple potential deadlock messages running komodod with DEBUG_LOCKORDER debugging macro on:
First issue originates in in MSG_BLOCK processing:

POTENTIAL DEADLOCK DETECTED
Previous lock order was:
 (1) pnode->cs_vSend  net.cpp:1629 (TRY)
 cs_main  main.cpp:8649 (TRY)
 cs_wallet  wallet/wallet.cpp:3222
 (2) cs_mapRelay  net.cpp:1924
Current lock order is:
 pnode->cs_vRecvMsg  net.cpp:1610 (TRY)
 cs_main  main.cpp:7355
 (2) cs_mapRelay  main.cpp:7449
 (1) cs_vSend  net.cpp:2218
Assertion failed: (onlyMaybeDeadlock), function potential_deadlock_detected, file sync.cpp, line 127.

For the first issue there is a discussion and a fix in another repo: dimxy#71

The second issue is supposedly in "getheaders" processing:

POTENTIAL DEADLOCK DETECTED
Previous lock order was:
 pnode->cs_vRecvMsg  net.cpp:1590 (TRY)
 (1) cs_main  main.cpp:8080
 (2) cs_vSend  net.cpp:2198
Current lock order is:
 (2) pnode->cs_vSend  net.cpp:1609 (TRY)
 (1) cs_main  main.cpp:8710 (TRY)
 (1) cs_main  main.cpp:2551

The second issue is still being investigated.

TestBlockValidity() call must be surrounded by cs_main

Inside the TestBlockValidity function there is an assertion for help lock on cs_main
However there is a call to TestBlockValidity in miner.cpp where this is not fulfilled. Need to be added

[enhancement] help method availability

./komodo-cli help
error code: -28
error message:
Loading block index...

help should return response regardless of init state

Error Fetching the Zcash Parameters

Describe the issue

When running "./zcutil/fetch-params.sh" it appears as though its trying to pull the "sproute-proving.key" file from https://z.cash/downloads/sprout-proving.key however that file appears to now be decommissioned. What should be replacing this when pulling attempting to pull them?

Expected behaviour

successfully pulling the zCash Parameters.
Instead i get an error trying to fetch file due to name being incorrect. When taking a look it appears to be depreciated

looking at pull requests i saw a change to fetch-params-alt.sh but when running that i get checksum did NOT match.

Following directions to install from https://developers.komodoplatform.com/basic-docs/smart-chains/smart-chain-setup/installing-from-source.html#linux in case someone wants to update that doc as well

getblocktemplate close the node without errors in debug.log

Hello!

If we run node with options:

mineraddress=OURADDRESS
disablewallet=1
minetolocalwallet=0

After full synchronization rpc command "getinfo" works fine. But rpc command "getblocktemplate" close the node without errors in debug.log.

THROW_IF_SYNCING incorrectly disabled for what reason?

THROW_IF_SYNCING is an important mitigation against metadata leakage and it was recently disabled in 2 places and forgot to be disabled in the 3rd place.

Why was this done without asking for feedback on the privacy/security implications?

komodod crash in mining code

With mining on, komodod sometimes crashes (observed on a dedicated node, with no ASIC).
Here is the backtrace:

Core was generated by `./komodod -gen -genproclimit=1 -pubkey=...'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  BitcoinMiner (pwallet=<optimized out>) at miner.cpp:1624
1624                                    index_vector[i] = eq.sols[s][i];
[Current thread is 1 (Thread 0x7f442bfff700 (LWP 26388))]

It looks like that the index 's' ranging from 0 from eq.nsols may get over eq.sols upper bound equal to MAXSOLS.
To fix this, the suggestion is to have the range for 's' from 0 to min(MAXSOLS, eq.nsols)

Reduce (remove?) the use of recursive mutexes

The current codebase makes heavy use of mutexes. Most mutexes are recursive. As this comment in the codebase hints at, and many online discussions talk about, recursive mutexes should be avoided.

Switching from a recursive mutex in some parts of the codebase will be relatively straightforward and easy. Other areas will require some thought and changes to method signatures.

TL;DR> With a bit of forethought and a good dose of effort, we can modify the existing codebase to make locking safer and performant.

I have included here an example of a typical (not easy, not hard) use case. As is obvious, each case will be different.

The method CheckProofOfWork is passed the CBlockHeader to check, a pubkey33, the height, and the consensus params. From there, it grabs some notary information and the current tip time. To grab that information requires a (read) lock on cs_main. Sometimes, the caller already has the lock, and other times it does not. Using a recursive lock, we just lock it, and get the information we need, making sure to "unlock" it on the way out.

But as we read in the links above, recursive locks should be avoided. And they can be avoided in this case fairly easily. CheckProofOfWork will not be harmed if "stale" data is used. Even a chain reorg in another process shouldn't cause a problem within this method. So the basic responsibility of CheckProofOfWork should not require that cs_main be locked.

However, the current implementation of this method requires that lock to safely get information from chainActive. So we have a few choices here.

1. Use recursive locking as we do now.
2. Require that the caller always has the lock held before calling this method. We will get the information from chainActive without locking it.
3. Require that the caller never has the lock held before calling this method. We will lock chainActive when we need to.
4. Require that the caller pass us a boolean as to whether cs_main is locked or not, or have two methods that do the same thing.
5. Require that the caller gather the information from chainActive that we need and pass it to the method, so CheckProofOfWork is lock-free.

These are my opinions, numbered to match the options above:

1. Recursive locking should be avoided.
2. Having the caller lock it means some processing will be done with the lock needlessly held, slowing down other threads.
3. This is a viable option IF we can guarantee that this method will not be part of a process that IS sensitive to stale data.
4. Now we are doing conditional locking. In my mind, this is more prone to errors than option 1.
5. Although CheckProofOfWork does not seem to be affected by stale data, it could be involved in a process that is sensitive. Having the caller gather the information using proper locking, and then passing that info to our method means stale-sensitive processes can hold the lock, and performance-sensitive processes don't need to.

We have weapons that will help us in our fight.

1. There are existing runtime checks (incompletely implemented). Running the code in a test environment with the right compile flags will help diagnose problems.
2. There are existing static checks (incompletely implemented). Warnings at compile time will help prevent problems.

To move forward, I believe we need to agree that such modifications are worth the effort. Then I believe we should assign a priority to it. Migrating away from recursive locks can be done in steps, and IMO is a safer way to handle the task.

rm `test_block_ser`

As all changes have been verified on the accepting to the dev branch stage, or within hfnet testing, and passing through dev->beta->master haven't new changes -> LGTM.

p.s. test_block_ser in TestHarakaRemoval tests could be safely deleted.

Originally posted by @DeckerSU in #544 (review)

A request to get an on-chain dataset

I am working on a research on the reasons why the atomic swap of the blockchain fails, but in fact I cannot obtain a valid real-case data set for analysis, I would like to ask where to get similar data to study, hope to cooperate with you, thank you.

gcc9 compilation error

test-komodo/test_eval_notarisation.cpp:59:25: error: non-local lambda expression cannot have a capture-default

Arch Linux compile fail with error: ‘invalid_argument’ is not a member of ‘std’ (FIXED WITH patch)

Describe the issue
Compile fail with error:

./crypto/equihash.h:256:20: error: ‘invalid_argument’ is not a member of ‘std’
  256 |         throw std::invalid_argument("Unsupported Equihash parameters");
      |                    ^~~~~~~~~~~~~~~~
./crypto/equihash.h: In function ‘bool EhOptimisedSolve(unsigned int, unsigned int, const eh_HashState&, std::function<bool(const std::vector<unsigned char>&)>, std::function<bool(EhSolverCancelCheck)>)’:
./crypto/equihash.h:284:20: error: ‘invalid_argument’ is not a member of ‘std’
  284 |         throw std::invalid_argument("Unsupported Equihash parameters");
      |                    ^~~~~~~~~~~~~~~~
make[2]: *** [Makefile:2808: crypto/libbitcoin_crypto_a-equihash.o] Error 1

Can you reliably reproduce the issue?
If so, please list the steps to reproduce below:

./zcutil/build.sh -j 7

make[3]: Entering directory '/home/luca/CC/Komodo/komodo/src/leveldb'
/home/luca/CC/Komodo/komodo/depends/x86_64-unknown-linux-gnu/share/../native/bin/ccache g++ -m64 -std=c++11 -I. -I./include -std=c++0x  -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -Wformat -Wformat-security -Wstack-protector -fstack-protector-all -fPIE   -pipe -O1 -g -fopenmp -fwrapv -fno-strict-aliasing -Wno-builtin-declaration-mismatch -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -I/home/luca/CC/Komodo/komodo/depends/x86_64-unknown-linux-gnu/share/../include/libsnark -I/home/luca/CC/Komodo/komodo/depends/x86_64-unknown-linux-gnu/share/../include/  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS -DMULTICORE -D__STDC_LIMIT_MACROS -c helpers/memenv/memenv.cc -o helpers/memenv/memenv.o
mv -f .deps/libbitcoin_util_a-utilmoneystr.Tpo .deps/libbitcoin_util_a-utilmoneystr.Po
make  -C secp256k1 libsecp256k1.la OPTFLAGS="-O2 -march=x86-64 -g "
make[3]: Entering directory '/home/luca/CC/Komodo/komodo/src/secp256k1'
gcc -I. -g -O2 -Wall -Wextra -Wno-unused-function -c src/gen_context.c -o gen_context.o
/home/luca/CC/Komodo/komodo/depends/x86_64-unknown-linux-gnu/share/../native/bin/ccache g++ -m64 -std=c++11 -o libsnark/algebra/curves/alt_bn128/alt_bn128_g2.o   libsnark/algebra/curves/alt_bn128/alt_bn128_g2.cpp -c -MMD  -Wformat -Wformat-security -Wstack-protector -fstack-protector-all -fPIE  -fPIC -DBINARY_OUTPUT -DNO_PT_COMPRESSION=1 -fstack-protector-all -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wno-comment -Wfatal-errors -O2 -march=x86-64 -DMONTGOMERY_OUTPUT -DCURVE_ALT_BN128 -I/home/luca/CC/Komodo/komodo/depends/x86_64-unknown-linux-gnu/share/../include -Ilibsnark -DNO_PROCPS -static -DSTATIC -DMULTICORE -fopenmp
In file included from crypto/equihash.cpp:20:
./crypto/equihash.h: In function ‘bool EhBasicSolve(unsigned int, unsigned int, const eh_HashState&, std::function<bool(const std::vector<unsigned char>&)>, std::function<bool(EhSolverCancelCheck)>)’:
./crypto/equihash.h:256:20: error: ‘invalid_argument’ is not a member of ‘std’
  256 |         throw std::invalid_argument("Unsupported Equihash parameters");
      |                    ^~~~~~~~~~~~~~~~
./crypto/equihash.h: In function ‘bool EhOptimisedSolve(unsigned int, unsigned int, const eh_HashState&, std::function<bool(const std::vector<unsigned char>&)>, std::function<bool(EhSolverCancelCheck)>)’:
./crypto/equihash.h:284:20: error: ‘invalid_argument’ is not a member of ‘std’
  284 |         throw std::invalid_argument("Unsupported Equihash parameters");
      |                    ^~~~~~~~~~~~~~~~
make[2]: *** [Makefile:2808: crypto/libbitcoin_crypto_a-equihash.o] Error 1

Machine specs:
OS name + version: Arch Linux
CPU: Intel
RAM: 16Gb
Disk size: 3Tb
Disk Type (HD/SDD): HD
Linux kernel version (uname -a): Linux miner0 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux
Compiler version (gcc --version): gcc (GCC) 10.2.0

A long delay in CAddrMan::Select() causes p2p messages to be blocked

There is a problem with addrman locks in komodod.

A delay exists in addrman.Select() function (

komodo/src/addrman.cpp

Line 361 in 9df2abc

const int kRetrySleepInterval = 100; // milliseconds

), which locks the addrman object for a long time (up to 2-3 min). So when the addrman is locked by Select() and if getaddr p2p cmd tries to lock addrman then p2p processing stops for a long time.

This is most noticeable for nspv clients which might timeout waiting for response from a nspv server.

The reason for this delay is explained in this zcash fix zcash#929.
However I checked the current bitcoin code, they do not seem to have such delays in addrman. Suggest removing it.
The addman.Select() is called quite regularly when the node is trying to maintain connections to peers.

A technique to access cc transactions in the connecting block

In cc modules a tx could spend another tx while it is staying in mempool, so when those two txns are added to the chain they are put in the same block.
So in the cc validation code we should provide that functions like myGetTransaction or CCgetspenttxid could access txns in the connecting block (when the cc validation code is run).
Currently this task is done as follows:
before a block is connected all mempool txns are removed and stored in a temp object, then mempool is filled with the block's transactions, the original mempool state is restored, the block is validated.
There are several issues with this approach:

• we need to lock mempool for some time what affects performance
• currently sometimes mempool cleanup is done inaccurately that creates situations when a tx is added in a block and stays in the mempool at the same time
• if the mempool is mixed with the already stored txns and the block's txns, this might affect functions checking mempool in the validation code. So we must clean the mempool completely and fill it with only block txns during the block validation and therefore we need to lock mempool even for longer time.

There is another experimental implementation of this feature, in an additional repository (we worked out it with decker).
This implementation does not change the mempool state at all but instead has a special version of myGetTransaction (eval->GetTxUnconfirmed) for use inside validation which accesses block txns directly (instead of mempool) when the block is connected.
This solution does not change the mempool state so no performance issues and no need to cleanup. Also there is no side effects if the validation code checks if a tx is spent in mempool (as the mempool state might be different on different nodes).
What we need to do for this case is make special functions for use inside the validation code that will look into connecting block transactions instead of mempool.
(would like to get opinions, @ca333, @Alrighttt, @jl777, @DeckerSU)

extensible data format for Antara modules

@Mixa84 @dimxy @ca333 @jl777
We need to design an extensible format for storing arbitrary data for arbitrary condition structures that can be stored within an OP_CHECKCRYPTOCONDTION scriptpubkey, presumably as OP_DROP data.

What I mean by this is that...

If we have the following condition structure:

{
    "type": "threshold-sha-256",
    "threshold":    1,
    "subfulfillments":  [
    {
        "type": "secp256k1-sha-256",
        "publicKey":    "02C03CDB8822DAA30241CC137FBBEE76EAD55D7F92E667DF1C8CE56E6E50824386"
    }, 
	{
        "type": "threshold-sha-256",
        "threshold":    5,
        "subfulfillments":  [
	        {
	            "type": "eval-sha-256",
	            "code": "force_txfee(x, y)"
	        },
	        {
	            "type": "eval-sha-256",
	            "code": "must_pay_p2pk(w, x, y, z)"
	        },
	        {
	            "type": "eval-sha-256",
	            "code": "must_pay_CC(x, y, z)"
	        },
	        {
	            "type": "eval-sha-256",
	            "code": "minmax_vinsvouts(x,x,y,y)"
	        },
	        {
	            "type": "eval-sha-256",
	            "code": "minmax_CCvinsvouts(y,y,y,y)"
	        }
        ]
        }
}

With the current implementation of cryptoconditions library, this structure will be hashed, but the arguments for each cannot be included in this hash.

So instead, we will remove all the arguments, hash the structure and this hash is used with OP_CHECKCRYPTOCONDITION in a scriptPubKey. Now the issue is that when this scriptPubKey needs to be spent, we know the eval codes from the scriptsig, but we do not know the arguments to provide to each.

I am proposing that we store the arguments inside the scriptPubKey via an OP_DROP(or many as each is limited to 520 bytes) appended to the end of the typical OP_CHECKCRYPTOCONDITION. Before we can begin writing modules, I believe we need to agree on a format that can handle any situation.

I am propsing we use the same BER encoding that we use for cryptocondition fullfillments and scriptsigs. As an example, the above condition structure's fullfillment(and scriptsig) would look something like this decoded(NOT EXACT)

[2] (2 elem)
  [0] (1 elem)
    [5] (2 elem)
      [0] (33 byte) 03682B255C40D0CDE8FAEE381A1A50BBB89980FF24539CB8518E294D3A63CEFE12
      [1] (64 byte) 598E1901B424A70C48047213E14BAF27DBEC0044E508CD8FE3CCC9A5C95C8D84652866…
  [1] (1 elem)
    [2] (5 elem)
      [0] (1 byte) 01
      [1] (1 byte) 02
      [2] (1 byte) 03
      [2] (1 byte) 04
      [2] (1 byte) 05

So, I am proposing we follow this same structure in order to facilitate arbitrary structures and being able to "route" arbitrary arguments to their respective eval codes.

[2] (2 elem)
  [0] (1 elem)
    [5] (2 elem) <DUMMY for secp256k1 node>
      [0] (0 byte) 
      [1] (0 byte) 
  [1] (1 elem)
    [2] (5 elem)
      [0] [1] (2 elem)
      	[0] (1 byte) x
      	[1] (1 byte) y
      [1] [1] (4 elem)
      	[0] (1 byte) w
      	[1] (1 byte) x
       	[2] (1 byte) y
      	[3] (1 byte) z
      [2] [1] (3 elem)
        [0] (1 byte) x
       	[1] (1 byte) y
      	[2] (1 byte) z
      [3] [1] (4 elem)
      	[0] (1 byte) x
      	[1] (1 byte) x
       	[2] (1 byte) y
      	[3] (1 byte) y
      [3] [1] (4 elem)
      	[0] (1 byte) y
      	[1] (1 byte) y
       	[2] (1 byte) y
      	[3] (1 byte) y

The idea is that we always use the same "index"(not sure this is the correct term) in the data structure as we do in the condition structure for the corresponding eval node. This should mean that it is extensible and no matter the condition structure, we know exactly where to look in the OP_DROP data structure for the arguments. The exact data structure can be defined by each individual module. So for example, a module might need a mask and a string to be passed to it. It could simply expect this to be passed to it and the OP_DROP data for that particular eval node would look something like:

[3] [1] (2 elem)
	[0] (1 byte) cc
	[1] (8 byte) DEADBEEF

I think it would be wise to use a well known structure such as BER as we can utilize already written libraries across many different languages.

I could use help designing this in C++, and I need someone to create helper functions.

I can design this in rust(and extend it via pyCC to python eventually).

I appreciate feedback from anyone, not just the people tagged, thanks.

komodod with mining on sometimes appears to hang

komodod executable sometimes hangs with its rpcs becoming not responsible. No crash happens though, the process remains in memory and the only option is to kill it.
The backtrace analysis with gdb debugger shows that it seems to be a deadlock:
threads 24 and 23 are waiting on two different critical sections (cs) but appear to have already made locks on those cs mutually:

Thread 23 (processing incoming messages) is waiting on the pqueue->ControlMutex in CCheckQueueControl::CCheckQueueControl() and it should have made a lock on cs_main already in ActivateBestChain():

Thread 23 (Thread 0x7fcc36ffd700 (LWP 7827)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007fcced9e7025 in __GI___pthread_mutex_lock (
    mutex=mutex@entry=0x561e336a61c8 <scriptcheckqueue+264>)
    at ../nptl/pthread_mutex_lock.c:80
#2  0x0000561e3254c186 in boost::posix::pthread_mutex_lock (
    m=0x561e336a61c8 <scriptcheckqueue+264>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/pthread_helpers.hpp:79
#3  boost::mutex::lock (this=0x561e336a61c8 <scriptcheckqueue+264>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/mutex.hpp:61
#4  CCheckQueueControl<CScriptCheck>::CCheckQueueControl (
    pqueueIn=0x561e336a60c0 <scriptcheckqueue>, this=0x7fcc36ffa980) at checkqueue.h:203
#5  ConnectBlock (block=..., state=..., pindex=<optimized out>, 
    pindex@entry=0x7fcc2417f210, view=..., fJustCheck=fJustCheck@entry=false, 
    fCheckPOW=fCheckPOW@entry=true) at main.cpp:3543
#6  0x0000561e32554fd3 in ConnectTip (pblock=0x7fcc36ffbda0, pindexNew=0x7fcc2417f210, 
    state=...) at main.cpp:4273
#7  ActivateBestChainStep (fSkipdpow=fSkipdpow@entry=false, state=..., 
    pindexMostWork=pindexMostWork@entry=0x7fcc2417f210, pblock=<optimized out>)
    at main.cpp:4551
#8  0x0000561e32556d99 in ActivateBestChain (fSkipdpow=fSkipdpow@entry=false, state=..., 
    pblock=pblock@entry=0x7fcc36ffbda0) at main.cpp:4612
---Type <return> to continue, or q <return> to quit---
#9  0x0000561e32557608 in ProcessNewBlock (from_miner=from_miner@entry=false, 
    height=height@entry=0, state=..., pfrom=pfrom@entry=0x7fcc2400d8a0, 
    pblock=pblock@entry=0x7fcc36ffbda0, fForceProcessing=<optimized out>, dbp=0x0)
    at main.cpp:5818
#10 0x0000561e3255d5cc in ProcessMessage (pfrom=pfrom@entry=0x7fcc2400d8a0, 
    strCommand="block", vRecv=..., nTimeReceived=nTimeReceived@entry=1657804006280505)
    at main.cpp:8001
#11 0x0000561e3255f472 in ProcessMessages (pfrom=0x7fcc2400d8a0) at main.cpp:8248
#12 0x0000561e32561b86 in boost::detail::function::function_invoker1<bool (*)(CNode*), bool, CNode*>::invoke (function_ptr=..., a0=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/function/function_template.hpp:100
#13 0x0000561e325d55bf in boost::function1<bool, CNode*>::operator() (a0=<optimized out>, 
    this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/function/function_template.hpp:764
#14 boost::signals2::detail::call_with_tuple_args<bool>::m_invoke<boost::function<bool (CNode*)>, 0u, CNode*&>(boost::function<bool (CNode*)>&, boost::signals2::detail::unsigned_meta_array<0u>, std::tuple<CNode*&> const&, boost::disable_if<boost::is_void<boost::function<bool (CNode*)>::result_type>, void>::type*) const (args=std::tuple containing = {...}, 
    func=..., this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/signals2/detail/variadic_slot_invoker.hpp:98
#15 boost::signals2::detail::call_with_tuple_args<bool>::operator()<boost::function<bool (CNode*)>, CNode*&, 1ul>(boost::function<bool (CNode*)>&, std::tuple<CNode*&> const&, mpl_::size_t<1ul>) const (args=std::tuple containing = {...}, func=..., this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/signals2/detail/variadic_slot_invoker.hpp:90
#16 boost::signals2::detail::variadic_slot_invoker<bool, CNode*>::operator()<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >(boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > const&) const (connectionBody=..., this=0x7fcc36ffcb30)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/signals2/detail/variadic_slot_invoker.hpp:134
#17 boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> >::dereference() const (this=0x7fcc36ffc6f0)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/signals2/detail/slot_call_iterator.hpp:110
#18 boost::iterators::iterator_core_access::dereference<boost::signals2::detail::slot_call_i---Type <return> to continue, or q <return> to quit---
terator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >(boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > const&) (f=...)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/iterator/iterator_facade.hpp:550
#19 boost::iterators::detail::iterator_facade_base<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> >, bool, boost::iterators::single_pass_traversal_tag, bool const&, long, false, false>::operator*() const (
    this=0x7fcc36ffc6f0)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/iterator/iterator_facade.hpp:656
#20 CombinerAll::operator()<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >(boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> >, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> >) const (last=..., first=..., this=<optimized out>) at net.h:111
#21 boost::signals2::detail::combiner_invoker<bool>::operator()<CombinerAll, boost::signals2---Type <return> to continue, or q <return> to quit---
::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >(CombinerAll&, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> >, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*), boost::function<bool (CNode*)> >, boost::signals2::mutex> >) const (
    last=..., first=..., combiner=..., this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/signals2/detail/result_type_wrapper.hpp:53
#22 boost::signals2::detail::signal_impl<bool (CNode*), CombinerAll, int, std::less<int>, boost::function<bool (CNode*)>, boost::function<bool (boost::signals2::connection const&, CNode*)>, boost::signals2::mutex>::operator()(CNode*) (args#0=<optimized out>, 
    this=0x561e35623a60)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/signals2/detail/signal_template.hpp:247
#23 boost::signals2::signal<bool (CNode*), CombinerAll, int, std::less<int>, boost::function<bool (CNode*)>, boost::function<bool (boost::signals2::connection const&, CNode*)>, boost::signals2::mutex>::operator()(CNode*) (args#0=<optimized out>, 
    this=0x561e336f8118 <g_signals+24>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/signals2/detail/signal_template.hpp:722
#24 ThreadMessageHandler () at net.cpp:1604
#25 0x0000561e324f0f45 in TraceThread<void (*)()> (name=<optimized out>, 
    func=0x561e325d4f0b <ThreadMessageHandler()>) at ./util.h:272
#26 0x0000561e324eae99 in boost::_bi::list2<boost::_bi::value<char const*>, boost::_bi::value<void (*)()> >::operator()<void (*)(char const*, void (*)()), boost::_bi::list0> (
    a=<synthetic pointer>..., f=<optimized out>, this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/bind/bind.hpp:319
#27 boost::_bi::bind_t<void, void (*)(char const*, void (*)()), boost::_bi::list2<boost::_bi::value<char const*>, boost::_bi::value<void (*)()> > >::operator() (this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/bind/bind.hpp:1294
#28 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(char const*, void (*)()), b---Type <return> to continue, or q <return> to quit---
oost::_bi::list2<boost::_bi::value<char const*>, boost::_bi::value<void (*)()> > > >::run (
    this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/detail/thread.hpp:120
#29 0x0000561e32baa43c in thread_proxy ()
#30 0x00007fcced9e46db in start_thread (arg=0x7fcc36ffd700) at pthread_create.c:463
#31 0x00007fccecd2461f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 24 (mining thread) is waiting on cs_main but it should have made a lock on pqueue->ControlMutex in ConnectBlock():

Thread 24 (Thread 0x7fcc367fc700 (LWP 7828)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007fcced9e70f4 in __GI___pthread_mutex_lock (
    mutex=mutex@entry=0x561e336a5a20 <cs_main>) at ../nptl/pthread_mutex_lock.c:115
#2  0x0000561e324f58b9 in boost::posix::pthread_mutex_lock (m=0x561e336a5a20 <cs_main>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/pthread_helpers.hpp:79
#3  boost::recursive_mutex::lock (this=0x561e336a5a20 <cs_main>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/recursive_mutex.hpp:108
#4  AnnotatedMixin<boost::recursive_mutex>::lock (this=0x561e336a5a20 <cs_main>)
    at sync.h:76
#5  boost::unique_lock<AnnotatedMixin<boost::recursive_mutex> >::lock (this=0x7fcc367f6740)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/lock_types.hpp:346
#6  CMutexLock<AnnotatedMixin<boost::recursive_mutex> >::Enter (nLine=<optimized out>, 
    pszFile=<optimized out>, pszName=<optimized out>, this=0x7fcc367f6740) at sync.h:132
#7  CMutexLock<AnnotatedMixin<boost::recursive_mutex> >::CMutexLock (this=0x7fcc367f6740, 
    mutexIn=..., pszName=<optimized out>, pszFile=<optimized out>, nLine=<optimized out>, 
    fTry=<optimized out>) at sync.h:153
#8  0x0000561e3251c94e in GetSpendHeight (inputs=...) at main.cpp:2764
#9  0x0000561e32526afd in ContextualCheckInputs (tx=..., state=..., inputs=..., 
    fScriptChecks=<optimized out>, flags=flags@entry=513, 
    cacheStore=cacheStore@entry=false, txdata=..., consensusParams=..., 
    consensusBranchId=1991772603, pvChecks=0x7fcc367f6b70) at main.cpp:2884
#10 0x0000561e3254d440 in ConnectBlock (block=..., state=..., pindex=<optimized out>, 
    pindex@entry=0x7fcc367f71d0, view=..., fJustCheck=fJustCheck@entry=true, 
    fCheckPOW=fCheckPOW@entry=true) at main.cpp:3676
#11 0x0000561e32560be9 in TestBlockValidity (state=..., block=..., 
    pindexPrev=0x7fcc248a1e30, fCheckPOW=fCheckPOW@entry=true, 
    fCheckMerkleRoot=fCheckMerkleRoot@entry=false) at main.cpp:5853
#12 0x0000561e325b58cb in <lambda(std::vector<unsigned char, std::allocator<unsigned char> >)>::operator()(std::vector<unsigned char, std::allocator<unsigned char> >) const (
    __closure=0x7fcc1875c480, soln=std::vector of length 1344, capacity 1344 = {...})
    at miner.cpp:1564
#13 0x0000561e325b6829 in std::_Function_handler<bool(std::vector<unsigned char, std::allocator<unsigned char> >), BitcoinMiner(CWallet*)::<lambda(std::vector<unsigned char, std::allocator<unsigned char> >)> >::_M_invoke(const std::_Any_data &, std::vector<unsigned char, std::allocator<unsigned char> > &&) (__functor=..., __args#0=...)
---Type <return> to continue, or q <return> to quit---
    at /usr/include/c++/7/bits/std_function.h:302
#14 0x0000561e325c2fee in std::function<bool (std::vector<unsigned char, std::allocator<unsigned char> >)>::operator()(std::vector<unsigned char, std::allocator<unsigned char> >) const (this=this@entry=0x7fcc367f8240, __args#0=std::vector of length 0, capacity 0)
    at /usr/include/c++/7/bits/std_function.h:706
#15 0x0000561e325befd3 in BitcoinMiner (pwallet=<optimized out>) at miner.cpp:1628
#16 0x0000561e325bfd0c in boost::_bi::list1<boost::_bi::value<CWallet*> >::operator()<void (*)(CWallet*), boost::_bi::list0> (a=<synthetic pointer>..., f=<optimized out>, 
    this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/bind/bind.hpp:259
#17 boost::_bi::bind_t<void, void (*)(CWallet*), boost::_bi::list1<boost::_bi::value<CWallet*> > >::operator() (this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/bind/bind.hpp:1294
#18 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(CWallet*), boost::_bi::list1<boost::_bi::value<CWallet*> > > >::run (this=<optimized out>)
    at /home/ubuntu/komodo/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/detail/thread.hpp:120
#19 0x0000561e32baa43c in thread_proxy ()
#20 0x00007fcced9e46db in start_thread (arg=0x7fcc367fc700) at pthread_create.c:463
#21 0x00007fccecd2461f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

It looks like the root of the issue is that there is lack of synchronisation of those two threads entering into ConnectBlock() so they create a dead lock. It seems there is a missing lock on the cs_main critical section before TestBlockValidity call in the ValidBlock lambda function in miner.cpp, this lock would provide the needed synchronisation.

there is a related issue #483

Ubuntu 22 crashes

This issue tracker is only for technical issues related to komodod

General Komodo questions and/or support requests and are best directed to Discord

Describe the issue

Please provide a general summary of the issue you're experiencing

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

1. yes on reboot and restart
2. crashes after Ubuntu 22 upgrade
3. komodod[4090]: segfault at 18 ip 000055d76b92bd53 sp 00007ffffea7aa70 error 4 in komodod[55d76b8e2000+c68000]
4. multiple servers
5. cleaning and rebuilding now after upgrade to see if new compiler & libararies fix problem
6. first clean and make rusults: failed

sudo make install
[sudo] password for doug:
Making install in src
make[1]: Entering directory '/home/doug/github/komodo/src'
make[2]: Entering directory '/home/doug/github/komodo/src'
  CXXLD    komodod
/usr/bin/ld: /home/doug/github/komodo/depends/x86_64-unknown-linux-gnu/lib/libgmp.a(dive_1.o): warning: relocation against `__gmp_binvert_limb_table' in read-only section `.text'
/usr/bin/ld: /home/doug/github/komodo/depends/x86_64-unknown-linux-gnu/share/../lib/libdb_cxx-6.2.a(os_yield.o): in function `__os_yield':
os_yield.c:(.text+0xce): undefined reference to `pthread_yield'
/usr/bin/ld: /home/doug/github/komodo/depends/x86_64-unknown-linux-gnu/lib/libevent.a(evutil_rand.o): in function `arc4_stir':
evutil_rand.c:(.text+0x250): undefined reference to `sysctl'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:2469: komodod] Error 1
make[2]: Leaving directory '/home/doug/github/komodo/src'
make[1]: *** [Makefile:5698: install-recursive] Error 1
make[1]: Leaving directory '/home/doug/github/komodo/src'
make: *** [Makefile:659: install-recursive] Error 1

7. rebuilding from fresh clone repository.

Actual behaviour + errors

Tell us what happens instead including any noticable error output (any messages displayed on-screen when e.g. a crash occurred)

komodod[4090]: segfault at 18 ip 000055d76b92bd53 sp 00007ffffea7aa70 error 4 in komodod[55d76b8e2000+c68000]

The version of Komodo you were using:

latest master & dev

Machine specs:

• OS Ubuntu 22.04.1 LTS

• CPU: Architecture: x86_64
  CPU op-mode(s): 32-bit, 64-bit
  Address sizes: 36 bits physical, 48 bits virtual
  Byte Order: Little Endian
  CPU(s): 4
  On-line CPU(s) list: 0-3
  Vendor ID: GenuineIntel
  Model name: Intel(R) Core(TM) i5-3470T CPU @ 2.90GHz
  CPU family: 6
  Model: 58
  Thread(s) per core: 2
  Core(s) per socket: 2
  Socket(s): 1
  Stepping: 9
  CPU max MHz: 3600.0000

• RAM: 16G

• Disk size: 1TB

• Disk Type (HD/SDD): SDD

• Linux kernel version (uname -a):

• Linux 138-backup-2 5.15.0-53-generic #59-Ubuntu SMP Mon Oct 17 18:53:30 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

• Compiler version (gcc -version):

COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.3.0-1ubuntu122.04' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu122.04)

Any extra information that might be useful in the debugging process.

This includes the relevant contents of ~/.komodo/debug.log. You can paste raw text, attach the file directly in the issue or link to the
2022-11-26 23:59:40 Zcash version v0.7.2-beta1-d456be35a-dirty (2022-06-13 22:13:47 +0200)
2022-11-26 23:59:40 Using /16 prefix for IP bucketing
2022-11-26 23:59:40 Using the 'standard' SHA256 implementation
2022-11-26 23:59:40

2022-11-26 23:59:40 Komodo version v0.7.2-beta1-d456be35a-dirty (2022-06-13 22:13:47 +0200)
2022-11-26 23:59:40 Using OpenSSL version OpenSSL 1.1.1k 25 Mar 2021
2022-11-26 23:59:40 Using BerkeleyDB version Berkeley DB 6.2.23: (March 28, 2016)

When starting komodod with -datadir arg on Windows, it still try to fetch .conf from default datadir.

When starting komodod with custom data directory (set with -datadir= arg) on Windows, it still try to fetch komodo.conf from default datadir (AppData\Roaming\Komodo)
Which result in error:
error: incorrect rpcuser or rpcpassword (authorization failed)

tested with latest dev version 117dfd5 on windows 10.

add fixes for staking (from marmara experience)

there are several fixes made in marmara repo but have not yet been pulled into komodo.

1. static struct komodo_staking *array in komodo_staked() has a c++ CScript object, which destructor is not called when array element is deallocated with free(). Suggestion is to convert it to std::vector which calls destructors for elements and use its reserve() method instead of realloc()
2. in GenerateBitcoins() when mining/staking threads are interrupted they are not joined so 2 staking threads might be running thus overwriting the static struct komodo_staking *array. To prevent this it is suggested to add join() call after minerThreads->interrupt_all(); Also it is better to store struct komodo_staking *array as thread_local memory to guarantee non overriding it by several threads.
3. add boost::this_thread::interruption_point(); in the while loop in komodo_waituntilelegible() to allow earlier stop at requests.
   (see marmara repo for those changes)
4. remove marmara remains in komodo_bitcoind.h/cpp

CryptoConditions objects should know how to create themselves.

I will use an "Asset" as an example.

Currently the struct CCcontract_info is defined in CCinclude.h. This struct contains a number of generic fields for different contracts, as well as pointers to functions for validation and checking a CScript. To create an "Asset" contract object, requires the following (taken from faucet.cpp::FaucetInfo())

CCcontract_info C; 
CCcontract_info *cp = CCinit(&C,EVAL_FAUCET); 

This requires the creation of an object, and a method with a large case statement. In addition there are macros to help build the ismyvin function.

I believe all this to be unnecessary, and fairly easy to fix.

• CCcontract_info should be moved to CCcontract_info.h
• CCcontract_info will become a virtual struct which requires the definition of validate()
• CCAssetContract_info should be defined in CCassets.h, and derives from CCcontract_info, defining its own validate method (in CCassets.cpp probably)
• CCcustom.inc is no longer necessary, nor are the defines to make it work, as the ismyvin method can be moved into CCcontract_info. Any object-specific customizations can override this.
• Existing methods that take in a CCcontract_info pointer should not need modification.
• Classes that would like to take a subset of object types can overload methods. Invalid combinations could be caught a compile time.
• CCinit is no longer necessary and should be replaced by simply creating the object desired. New way:

CCAssetContract_info C;

some komodo_...() functions needs to be wrapped by cs_main lock

Several functions komodo_chainactive komodo_heightstamp komodo_chainactive_timestamp access the chainActive object that needs to be protected by cs_main critical section.
It looks like calls to these functions are not surrounded with LOC(cs_main) when required. At least the nspv server part implementation may call those functions without cs_main lock.
That needs to be added.
Also it is suggested to add AssertLockHeld(cs_main) into those functions to track locks (this is activated by macro DEBUG_LOCKORDER)

komodod CI py tests fail to run on Macos and Windows

On MacOs Test (MacOS/Dice, Token, Faucet, Rewards) and other fail with an error: src/pycurl.h:178:13: fatal error: 'openssl/ssl.h' file not found
if fix openssl/pycurl installation according to this https://stackoverflow.com/questions/72884308/error-openssl-crypto-h-file-not-found-when-installing-pycurl-on-mac-using-pip there is a new error: [Errno 8] Exec format error: '../../src/komodod'

On Windows same tests fail with errors:

ModuleNotFoundError: No module named 'wget'
C:\hostedtoolcache\windows\Python\3.[9](https://github.com/dimxy/komodo/actions/runs/3170816273/jobs/5164016290#step:5:10).13\x64\python.exe: No module named pytest
Error: Process completed with exit code 1.

Please port z_gettreestate RPC.

https://zcash-rpc.github.io/z_gettreestate.html

It is required for lightwalletd GetTreeState method (https://github.com/zcash/lightwalletd/blob/2d3943b8e995a3b2c5648ec9859dccc67c535386/walletrpc/service.proto#L177). With this, we can initialize PIRATE light wallet database with some recent block data in AtomicDEX, at least for newly created addresses. It will help to decrease first PIRATE activation time significantly. As of now, it may take 30 minutes (or even more) to scan all blocks data, searching for specific address transactions.

cc @ca333

valuePools (sprout, sapling) monitoring / calculation works incorrectly

Ever since we started using our own versioning system, which is similar to 0.7.x.y, instead of the default ZCash versioning 1.x.y.z (meaning our release has lower versions than the initial), it appears that we have encountered issues with monitoring the valuepools.

define(_CLIENT_VERSION_MAJOR, 0)
define(_CLIENT_VERSION_MINOR, 8)
define(_CLIENT_VERSION_REVISION, 1)
define(_CLIENT_VERSION_BUILD, 1)
...
static const int CLIENT_VERSION =
                           1000000 * CLIENT_VERSION_MAJOR
                         +   10000 * CLIENT_VERSION_MINOR
                         +     100 * CLIENT_VERSION_REVISION
                         +       1 * CLIENT_VERSION_BUILD;

For example, version 0.8.1.1 resulting in nVersion == 80101. And here:

komodo/src/chain.h

Lines 446 to 456 in d9144d0

	// Only read/write nSproutValue if the client version used to create
	// this index was storing them.
	if ((s.GetType() & SER_DISK) && (nVersion >= SPROUT_VALUE_VERSION)) {
	READWRITE(nSproutValue);
	}
	// Only read/write nSaplingValue if the client version used to create
	// this index was storing them.
	if ((s.GetType() & SER_DISK) && (nVersion >= SAPLING_VALUE_VERSION)) {
	READWRITE(nSaplingValue);
	}

We lost read/write of nSproutValue and nSaplingValue for the CDiskBlockIndex, as the values of SPROUT_VALUE_VERSION and SAPLING_VALUE_VERSION is higher than current actual version:

komodo/src/chain.h

Lines 40 to 41 in d9144d0

	static const int SPROUT_VALUE_VERSION = 1001400;
	static const int SAPLING_VALUE_VERSION = 1010100;

As a result this code for updating nChainSproutValue and nChainSaplingValue don't work, because pindex->nSproutValue and pindex->nSaplingValue are boost::none (no read from index):

komodo/src/main.cpp

Lines 6041 to 6094 in d9144d0

	for(const auto& item : vSortedByHeight)
	{
	CBlockIndex* pindex = item.second;
	pindex->nChainWork = (pindex->pprev ? pindex->pprev->nChainWork : 0) + GetBlockProof(*pindex);
	// We can link the chain of blocks for which we've received transactions at some point.
	// Pruned nodes may have deleted the block.
	if (pindex->nTx > 0) {
	if (pindex->pprev) {
	if (pindex->pprev->nChainTx) {
	pindex->nChainTx = pindex->pprev->nChainTx + pindex->nTx;
	if (pindex->pprev->nChainSproutValue && pindex->nSproutValue) {
	pindex->nChainSproutValue = *pindex->pprev->nChainSproutValue + *pindex->nSproutValue;
	} else {
	pindex->nChainSproutValue = boost::none;
	}
	if (pindex->pprev->nChainSaplingValue) {
	pindex->nChainSaplingValue = *pindex->pprev->nChainSaplingValue + pindex->nSaplingValue;
	} else {
	pindex->nChainSaplingValue = boost::none;
	}
	} else {
	pindex->nChainTx = 0;
	pindex->nChainSproutValue = boost::none;
	pindex->nChainSaplingValue = boost::none;
	mapBlocksUnlinked.insert(std::make_pair(pindex->pprev, pindex));
	}
	} else {
	pindex->nChainTx = pindex->nTx;
	pindex->nChainSproutValue = pindex->nSproutValue;
	pindex->nChainSaplingValue = pindex->nSaplingValue;
	}
	}
	// Construct in-memory chain of branch IDs.
	// Relies on invariant: a block that does not activate a network upgrade
	// will always be valid under the same consensus rules as its parent.
	// Genesis block has a branch ID of zero by definition, but has no
	// validity status because it is side-loaded into a fresh chain.
	// Activation blocks will have branch IDs set (read from disk).
	if (pindex->pprev) {
	if (pindex->IsValid(BLOCK_VALID_CONSENSUS) && !pindex->nCachedBranchId) {
	pindex->nCachedBranchId = pindex->pprev->nCachedBranchId;
	}
	} else {
	pindex->nCachedBranchId = SPROUT_BRANCH_ID;
	}
	if (pindex->IsValid(BLOCK_VALID_TRANSACTIONS) && (pindex->nChainTx || pindex->pprev == NULL))
	setBlockIndexCandidates.insert(pindex);
	if (pindex->nStatus & BLOCK_FAILED_MASK && (!pindexBestInvalid || pindex->nChainWork > pindexBestInvalid->nChainWork))
	pindexBestInvalid = pindex;
	if (pindex->pprev)
	pindex->BuildSkip();
	if (pindex->IsValid(BLOCK_VALID_TREE) && (pindexBestHeader == NULL || CBlockIndexWorkComparator()(pindexBestHeader, pindex)))
	pindexBestHeader = pindex;
	}

As a result getblockchaininfo RPC output looks like this:

  "valuePools": [
    {
      "id": "sprout",
      "monitored": false
    },
    {
      "id": "sapling",
      "monitored": true,
      "chainValue": 0.00000000,
      "chainValueZat": 0
    }
  ],

The offer is to change SPROUT_VALUE_VERSION and SAPLING_VALUE_VERSION to 80102 (0.8.1-beta3) in KomodoOcean and komodod simultaneously, otherwise the bootstraps between original komodod and KomodoOcean will be incompatible (of course, it assumes publishing releases 80102, 0.8.1-beta3)

Since monitoring value pools is not a critical issue, we can delay addressing it. However, if we decide to make changes now, both daemons in KomodoOcean and this repository should have the same version and the same read/write activation constants.

libsodium-1.0.18.tar.gz download error

Looks like download.libsodium.org have some errors in TLS 1.3 setup (you can check SSL setup on this host by https://www.cdn77.com/tls-test and it will give "enabled connection failed rather than downgrading to TLSv1.2" error), due to these errors wget during downloading libsodium-1.0.18.tar.gz will give the following error:

GnuTLS: received alert [70]: Error in protocol version.

The issue appeared at least on Debian 10 Buster (on Ubuntu 16.04 issue doesn't appear, probably bcz of old version OpenSSL and wget, which is not TLS 1.3 compatible). Two variants of fix is possible at this moment (changes needed in depends/builders/linux.mk), one with using curl as downloader for linux platform and second is continue to using wget, but forced to use TLS 1.2:

depends/builders/linux.mk changes:

Variant 1:

build_linux_DOWNLOAD = curl --location --fail --connect-timeout $(DOWNLOAD_CONNECT_TIMEOUT) --retry $(DOWNLOAD_RETRIES) -o

Variant 2:

build_linux_DOWNLOAD = wget --secure-protocol=TLSv1_2 --timeout=$(DOWNLOAD_CONNECT_TIMEOUT) --tries=$(DOWNLOAD_RETRIES) -nv -O

In Komodo-QT (KomodoOcean) we used Variant 1 to fix this error:
DeckerSU/KomodoOcean@86aba66

p.s. To reproduce the issue just try to make fresh clone of repo and build from scratch on Debian 10.

Proposal to use metadata with cryptoconditions

This is an idea how improve cryptoconditions usage in blockchain (a copy that I posted in our group chat).
Cryptoconditions is a technology allowing to verify that some cryptographic operation (like signature or hash) is true or not.
The reason for this improvement is that the original cryptoconditions technology is blockchain agnostic and we need to extend it for use in blockchains. The proposal is about adding some metadata to conditions with the info about condition's content (because the condition itself is a hash)

As a reminder, a cryptocondition is a pair of a condition and fulfilment.
The condition itself is a fingerprint (hash) stored in the scriptPubKey. There is a paired fulfilment in the spending scriptSig where the cryptocondition data are open to allow to evaluate it. This is similar to P2PKH, where a pubkey hash is stored in the scriptPubkey and the pubkey is stored in the spending scriptSig.
The cryptoconditions standard (https://tools.ietf.org/html/draft-thomas-crypto-conditions-04) stands that the fulfilment should not be derived from the condition (and this is the reason the condition is hashed and also this makes it compact)
But Alright's concept of generic validators and our past cc development experience showed that often we needed to know about the condition content. Most important thing was to know which evalcode was inside the condition to ensure that tx outputs received cc value for this evalcode.
For this we made the cryptocondition library version 2, which changes the condition serialisation to make evalcodes visible in conditions.

But now I think this was a limited solution:
first, we broke the standard (this is always not good to break standards: we might lose security and potential interoperability with others who use cryptoconditions, for example, Verus or else).
second, our past experience shows not only evalcodes are needed to see in conditions. Sometimes it is good to know the destination pubkey(s).

I propose a more generalised approach: instead of changing condition serialisation to non-standard and exposing only evalcodes, we could attach metadata to conditions that would tell about conditions' content. (And go back to the original standard condition structure supposed by the standard, that is, before cc v2)
This metadata will be optional and cc module dependant. First thing, we could put evalcodes into the metadata, but we could also place various other info, needed by a cc module, like target pubkeys (to verify that the tx output is sent to the appropriate address)

scriptPubKey: <condition><metadata: evalcode(s), pubkey(s), other info about condition>
scriptSig: <fulfilment>

Special case for the creation tx:
currently initial transactions with cryptoconditions v1 are not sent to the cc validation code because those txns do not have cc vins (with fulfilments with decoded data) and the consensus code has no idea which evalcodes are inside conditions.
To provide creation cc txns are validated we will require that creation txns will always have metadata with evalcodes.

The cc validation code must validate that the metadata matches to the condition (this validation will be possible on spending, when the fulfilment is available). I think we might call metadata as 'manifest' as the tx creator declares the condition content.

For metadata encoding we might use the same ASN.1 encoding used to encode cryptoconditions (it is basically a property-value schema).

This is similar to Alright's proposal (he suggests not fingerprint the condition at all) but I propose to preserve the original condition fingerprinting and use metadata instead.

TLDR: it seems cc v2 does not fully cover devs needs as it only opens evalcodes but we need more info plus cc v2 breaks the original cc standard. Instead, let's add metadata to conditions with info about conditions, needed by cc devs, including evalcodes, and implement standard ASN.1 encoding for this metadata.

Tokensv2 RPC `help` output issues

This issue tracker is only for technical issues related to komodod

General Komodo questions and/or support requests and are best directed to Discord

Describe the issue

With the new PR to the Research branch that added Tokenv2, the Tokens RPC help command now shows both sets of tokensv1 and tokensv2 RPC in the same list. The other CC's are split by v1 and a separate v2 section below that. It also looks as though some of the help RPC's don't show the v2 version. I double checked in the research branch and found the v2 version in there, so I'm not sure where the disconnect is.

For example:
tokenask numtokens tokenid price shows twice (once for v1, a second time for v2), but I found the code below
https://github.com/KomodoPlatform/komodo/blob/research/src/rpc/tokensrpc.cpp#L699

tokenid bidtxid also shows twice but I cannot find it within the rpc.cpp

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

1. komodo-cli -ac_name=TKLTEST help

Expected behaviour

The tokensv1 and tokensv2 sections should be separate like the Oracles v1/v2, for example.

Actual behaviour + errors

Nil

The version of Komodo you were using:

"KMDversion": "0.6.1",

Machine specs:

Not applicable

Any extra information that might be useful in the debugging process.

Not applicable

Do you have a backup of ~/.komodo directory and/or take a VM snapshot?

Not applicable

targetSymbol in Eval::ImportCoin() must be checked for non empty

targetSymbol field in the import transaction is compared to various values to determine the import type. One of the cases is the check against the ASSETCHAINS_SELFIMPORT chain var and treated as a gateway import tx. If both targetSymbol and ASSETCHAINS_SELFIMPORT are empty the code flow enters gateway import statements what it should not do.
Both ASSETCHAINS_SELFIMPORT targetSymbol should be checked as non-empty in Eval::ImportCoin() before allowing to go further in the import tx validation.
(should ASSETCHAINS_SELFIMPORT be checked to a specific value like "GATEWAY"?)

tokel: tokenv2ask rpc able to create invalid tx

The tokenv2ask rpc needs additional checks to prevent it from creating transactions that are incompatible with consensus code. The specific check is located here:

komodo/src/cc/assets.cpp

Line 460 in 87fb701

else if (unit_price <= ASSETS_NORMAL_DUST)

with an identical check for bids located here as well:

komodo/src/cc/assets.cpp

Line 266 in 87fb701

else if (unit_price <= ASSETS_NORMAL_DUST)

@dimxy

komodo-cli -ac_name=TKLTEST6 tokenv2createtokel ALRIGHT 0.0001 BUYME '{ "url":"<url-string>", "id":999, "royalty":777, "arbitrary":"deadbeefdeadbeef" }'
{
  "result": "success",
  "hex": "0400008085202f89018988ce47fa1a71d04a36277c01178d8869d528ee53c1f2bde170f796fb66243c0000000048473044022015c77559d7bf6252950998901fcd9991be1dafe58e4843ae4c0b8856d2b7869802201757803f93a06613374b6e55ddfc4de5750443a140577356aeadd171f05bbf2601ffffffff041027000000000000403e4da23ba00aa003800102af038001f5a12da22b802096fec31e85a06720706ef9214c9c8b2df26940aac250e1d80f23a772b18b5a4a810302040082020204cc1027000000000000693e4da23ba00aa003800102af038001f5a12da22b802026ca62cd1e723e37845ca2bc571fd41ea1092811d686c0ff9be9e0c74aa9659f810302040082020204cc270402f501012102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b375d06bf50500000000232102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b3ac0000000000000000596a4c56f563012102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b307414c5249474854054255594d4522f701020c3c75726c2d737472696e673e01fde70303fd09030408deadbeefdeadbeef00000000862100000000000000000000000000"
}

11:49:12:~ % komodo-cli -ac_name=TKLTEST6 tokenv2info 060def9d47f74fa4c2654ccc8e1950993d56b6b8f439fea1a386322a159db9dd
{
  "result": "success",
  "tokenid": "060def9d47f74fa4c2654ccc8e1950993d56b6b8f439fea1a386322a159db9dd",
  "owner": "02fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b3",
  "name": "ALRIGHT",
  "supply": 10000,
  "description": "BUYME",
  "data": "f701020c3c75726c2d737472696e673e01fde70303fd09030402eeff",
  "version": 1,
  "IsMixed": "yes",
  "height": 8382
}

11:50:08:~ % komodo-cli -ac_name=TKLTEST6 tokenv2ask 9000 060def9d47f74fa4c2654ccc8e1950993d56b6b8f439fea1a386322a159db9dd 0.00000001
{
  "hex": "0400008085202f89028988ce47fa1a71d04a36277c01178d8869d528ee53c1f2bde170f796fb66243c000000004847304402200f039c0034cfa8f12c3734cda7158cef4c4bad215f9b957bc036aa8ec78377330220561f52a7d3d36d64322050d88f58a3261055661984e52a308226c0be42c4200501ffffffffddb99d152a3286a3a1fe39f4b8b6563d9950198ecc4c65c2a44ff7479def0d06010000007b4c79a276a072a26ba067a565802102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b38140f1219b20b893ee6a8b4557e07582814a9e212eea2051e7bc09c0e51abf898bf757e8463c982cc472bda3b31e81bd06bb4e27fcee294d9317eb3d53118152e19ca100af038001f5a10001ffffffff0528230000000000006e434da240a00fa003800103af038001f5af038001f6a12da22b8020372291b16b9c8bab27def01f3d96e96f97f5ab14b1cab090dbe9296b70fd87bb810302040082020204cc270402f60101210345d2e7ab018619da6ed58ccc0138c5f58a7b754bd8e9a1a9d2b811c5fe72d467751027000000000000403e4da23ba00aa003800102af038001f6a12da22b80208cf1d7140b0b18bca52230aafc916bcd1c1ca2912366ec0dcc78d827f983ef87810302080082020204cce803000000000000693e4da23ba00aa003800102af038001f5a12da22b802026ca62cd1e723e37845ca2bc571fd41ea1092811d686c0ff9be9e0c74aa9659f810302040082020204cc270402f501012102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b375e092f50500000000232102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b3ac0000000000000000586a4c55f57401060def9d47f74fa4c2654ccc8e1950993d56b6b8f439fea1a386322a159db9dd31f6730101000000000000002102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b33ebe000000000000872100000000000000000000000000"
}

11:50:23:~ % komodo-cli -ac_name=TKLTEST6 sendrawtransaction 0400008085202f89028988ce47fa1a71d04a36277c01178d8869d528ee53c1f2bde170f796fb66243c000000004847304402200f039c0034cfa8f12c3734cda7158cef4c4bad215f9b957bc036aa8ec78377330220561f52a7d3d36d64322050d88f58a3261055661984e52a308226c0be42c4200501ffffffffddb99d152a3286a3a1fe39f4b8b6563d9950198ecc4c65c2a44ff7479def0d06010000007b4c79a276a072a26ba067a565802102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b38140f1219b20b893ee6a8b4557e07582814a9e212eea2051e7bc09c0e51abf898bf757e8463c982cc472bda3b31e81bd06bb4e27fcee294d9317eb3d53118152e19ca100af038001f5a10001ffffffff0528230000000000006e434da240a00fa003800103af038001f5af038001f6a12da22b8020372291b16b9c8bab27def01f3d96e96f97f5ab14b1cab090dbe9296b70fd87bb810302040082020204cc270402f60101210345d2e7ab018619da6ed58ccc0138c5f58a7b754bd8e9a1a9d2b811c5fe72d467751027000000000000403e4da23ba00aa003800102af038001f6a12da22b80208cf1d7140b0b18bca52230aafc916bcd1c1ca2912366ec0dcc78d827f983ef87810302080082020204cce803000000000000693e4da23ba00aa003800102af038001f5a12da22b802026ca62cd1e723e37845ca2bc571fd41ea1092811d686c0ff9be9e0c74aa9659f810302040082020204cc270402f501012102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b375e092f50500000000232102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b3ac0000000000000000586a4c55f57401060def9d47f74fa4c2654ccc8e1950993d56b6b8f439fea1a386322a159db9dd31f6730101000000000000002102fcc1c62627264905f87f9117e0cd703f6a44038eca792dce014fc89dbbf887b33ebe000000000000872100000000000000000000000000
error code: -26
error message:
16: mandatory-script-verify-flag-failed (Script evaluated without error but finished with a false/empty top stack element)

2021-07-28 15:50:32 cc RunCCEval() ERROR: CC Eval evalcode: EVAL_ASSETSV2 Invalid, reason: invalid or too low unit price

Komodod segmantaion fault on Alpine Linux

Describe the issue

I completely built Komodo from scratch on Alpine Linux. Everything works fine about building. But komodod is facing segmentation fault error and has no error log.

Can you reliably reproduce the issue?

1. Run any Alpine Linux v3.11
2. Install packages like below
   sudo apk add --update alpine-sdk m4 autoconf libtool ncurses-dev unzip git python3 libzmq
   sudo apk add build-base libzmq musl-dev python3-dev zeromq-dev
   sudo pip3 install pyzmq
   sudo pip3 install --upgrade pip
   sudo apk add zlib-dev wget gnutls-dev automake curl libsodium-dev linux-headers cargo
3. build Komodo as usual like below
   git clone https://github.com/komodoplatform/komodo --branch master --single-branch
   cd komodo
   ./zcutil/fetch-params.sh
   ./zcutil/build.sh -j$(expr $(nproc) - 1)
4. enter src folder
   cd src
5. run komodod
   ./komodod --version

Expected behaviour

Komodod should work as usual and show its version

Actual behaviour + errors

The version of Komodo you were using:

There is no version info because of error.

git logs

commit b998ca1 (HEAD -> master, tag: cd_release_6ad6bc2_dev, tag: cd_release_4db5591_dev, origin/master, origin/HEAD)
Merge: 1e1c631 e7e8145
Author: ca333 [email protected]
Date: Fri May 22 00:55:40 2020 +0200

Merge pull request #359 from KomodoPlatform/beta

bump deprecation height

commit e7e8145
Merge: c340cdd 7af1817
Author: ca333 [email protected]
Date: Thu May 21 21:51:48 2020 +0200

Merge pull request #358 from KomodoPlatform/dev

bump deprecation height

commit 7af1817
Merge: c227519 e68462b
Author: ca333 [email protected]

Machine specs:

• OS name + version: Alpine Linux v3.11
• CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
• RAM: 4 GB
• Disk size: 4 GB
• Disk Type (HD/SDD): SSD
• Linux kernel version (uname -a): Linux worker01 5.4.34-0-virt #1-Alpine SMP Wed, 22 Apr 2020 19:26:07 UTC x86_64 Linux
• Compiler version (gcc -version): gcc (Alpine 9.2.0) 9.2.0

There is no .komodo folder created yet. So there is no debug log.
You can use vagrant box at this address for reproducing this issue: https://app.vagrantup.com/cgops/boxes/alpine-3.12

Nodes do not restore last notarisation on start-up

There is code in komodod which constantly updates the latest notarisation transaction (and also remembers the last notarised height which it points to). This latest notarised height is used to prevent forks growing before it, to ensure protection against 51% attacks.
The fork prevention code checks if a fork is before the last notarised height and the old chain tip is over it then it would not allow such fork activation as the new best chain (and also rejects the new block).

The issue is that nodes do not load the latest notarisation transaction (nota) from HDD on startup and would get only future notas from the network.
Just after start, until receiving the next nota, such a node may receive blocks sent by a malicious node and activate an unnotarised branch as the best chain (forked from any height within 100 blocks from the tip). If malicious nodes have enough power (51% attack) an attacked node may never activate the actual notarised branch.
I believe this is not a real vulnerability for the whole dpow protected chain (as nodes never start all at the same time) but it could be a problem for a selected node (imagine if this is an SPV node).
To fix this we may add code to restore the last nota in the node memory on startup, by browsing blocks down from the chain tip.

(One might ask what if a node was off for some long period then even the nota restored from HDD would not be the actual last nota so the node needed to sync with peers. However when a node is syncing and queries randomly different peers it is not that risky to get attacked by some malicious nodes than when a node is fully in sync and any other node may announce new blocks to it)
@Alrighttt @DeckerSU @tonymorony

Crashing daemon

Ubuntu 16.04

Occasionally, daemon crashes leaving following in debug.log
Referring to master tree

2019-07-19 23:30:01 ERROR: AcceptToMemoryPool: tx inputs not found
2019-07-19 23:30:37 ERROR: ContextualCheckBlock: contains a non-final transaction
2019-07-20 19:49:19

Get rid of CWallet::OrderedTxItems and keep wallet transactions in memory always ordered

Currently transactions in wallet's memory are ordered on purpose in OrderedTxItems. Also each in call to OrderedTxItems a walletdb is created which may take thousands of milliseconds. This especially affects importprivkey rpc.
The fix should remove OrderedTxItems() method and always keep transactions ordered.
When implementing avoid creating CWalletDB inside loops.

See also zcash@31d49b0

ERROR: CTxMemPool::ReadFeeEstimates() on komodod start up

On start komodod prints a error ERROR: CTxMemPool::ReadFeeEstimates() up-version (109900) fee estimate file
this is from this code

        filein >> nVersionRequired >> nVersionThatWrote;
        if (nVersionRequired > CLIENT_VERSION)
            return error("CTxMemPool::ReadFeeEstimates(): up-version (%d) fee estimate file", nVersionRequired);

nVersionRequired is set to 109900 in WriteFeeEstimates and recently CLIENT_VERSION changed from 3000100 to 70100 so the condition nVersionRequired > CLIENT_VERSION now is true.
Suggestion: change nVersionRequired from 109900 to 70100

[Feature Request] Add support of domain names in "-addnode=" komodod parameters

Just had a thought if komodod supports domain name with the parameter -addnode=, but I think it does not.

It would be great to add the root DNS (i.e. example.com) and also any subdomain (i.e. node.example.com, joe.node.example.com, bob.node.example.com) support to komodod if it already doesn't support.

Having this feature will allow using Dynamic DNS as a -addnode= parameter value, which can use dynamically changing IP for Antara Smartchains.

Change CC Vout to make Eval code visible

We can't currently validate a CC VOUT, because it is a hash of an eval code which we need to validate, and a pubkey which we might not know.

Potential solutions:

1. Include the pubkey unhashed someplace so that the cc vout can be validated.
   Upsides: Don't need to change the vouts
   Downsides: It's 32/33 bytes of redundant weight, and it reveals the pubkey which is not always
   desirable.

2. Encode the fulfillment (without a sig) instead of the condition in the scriptpubkey.
   Upsides: Conceptual simplicity.
   Downsides: Not necessarily desirable from an implementation perspective, as CC expects to encounter a signature in the fulfillment, so you need to hack around that somehow.

3. Add OP_EVAL opcode to a regular p2pkh (p2pkh+eval), ie: OP_DUP OP_HASH160 {addresshash} OP_EQUALVERIFY OP_CHECKSIGVERIFY {evalcode} OP_EVAL. This will be a new Opcode which pops the evalcode from the stack and returns true or false. OP_CHECKSIG also needs to be switched out with OP_CHECKSIGVERIFY so it bails if the signature is wrong.
   Upsides: Conceptual simplicity, visible addresshash, don't actually need CC to sign.
   Downsides: ?

Thoughts?

Detect and handle corrupt or missing blocks or indexes

Situation:

When a node encounters a problem, the block files or index files can become corrupted or have incomplete data.

Weapons:

Corruption and missing data can be detected. Other nodes can provide the information the corrupted node lacks.

Objective:

Detect the problem and repair it before allowing the node to report that it is fully synchronized.

Tactics:

1. For the case of corrupted block files, the chain must be downloaded from the network starting from the point the corruption was detected.
2. For missing block files, the chain must be downloaded and the node re-indexed.
3. For corrupted or missing indexes, the problem must be accurately detected and the user must be prompted to restart the node with the -reindex option.

Where we are now:

Some situations of block or index corruption are detected, others are not.

As an example, having the node crash after writing the block to disk but before the index file is written will lead to missing transactions when the node restarts. The node does restart, but the data is inaccurate. Looking for the block returns nullptr and looking for a transaction that was within that block returns that the transaction does not exist.

Additional Information:

The function LoadIndexDB() does some checks to verify the integrity of the block files. This may be a good place to add additional checks to verify that the blocks and the index are synchronized.

Note: The attempt is to assist node operators when hardware/software issues make a mess within the persisted data on disk. Detecting malicious modification to distort data is not considered here.

Note: Having a block who's previous block does not exist may not be an indication of corruption. It is a valid (temporary) situation that must be planned for.

In my testing:

1. Having an incomplete index file (an entire entry about a block does not exist) is not detected, and the node starts with incomplete data.
2. Having a corrupted index file (data truncated off the end) is not detected, and the node starts with a shortened chain. I have yet to test to see if it re-syncs correctly.
3. Having an incomplete block file (an entire block does not exist but does exist in the index) is not detected, but would probably be a very rare occurrence. We could test for it, but we may not want to concentrate heavily on detecting/fixing it.
4. Having a corrupted block file (block cannot be de-serialized) is detected. I have yet to test what options are available for a node operator beyond a full re-sync.

ToDo:

• Verify the findings above are accurate for different combinations of corruption / missing data.
• Run tests on a multi-node chain to determine current abilities for recovering from corruption.

See also:

Bitcoin issue 19274

[testnet] genesis block doesn't accept during reindex

Steps to reproduce:

1. Compile komodod from latest dev branch, and launch it, for example, with the following params:

-ac_name=INDXFAIL --testnet

As you will see in debug.log, the genesis block will be accepted:

2023-03-01 17:25:14 Pre-allocating up to position 0x1000000 in blk00000.dat
2023-03-01 17:25:14 UpdateTip: new best=e46e999d6de5ef95427ea51b8242ef7949de22b340b6a484ff154b9991ec9452  height=0  log2_work=4.0874628  tx=1  date=2011-02-02 23:16:42 progress=1.000000  cache=0.0MiB(0tx)
2023-03-01 17:25:14 init message: Rewinding blocks if needed...
2023-03-01 17:25:14 init message: Verifying blocks...

2. Change indexes in the ~/.komodo/INDXFAIL/INDXFAIL.conf to force reindex, for example, add the following lines into config:

addressindex=1
timestampindex=1
spentindex=1

As the previous indexes was only with txindex=1 it will trigger reindex:

2023-03-01 17:26:55 Reindexing block file blk00000.dat...

3. And then you will see in debug.log:

2023-03-01 17:26:55 Waiting for genesis block to be imported...
2023-03-01 17:26:55 Reindexing block file blk00000.dat...
2023-03-01 17:26:55 ERROR: CheckBlock: proof of work failed
2023-03-01 17:26:55 InvalidChainFound: invalid block=e46e999d6de5ef95427ea51b8242ef7949de22b340b6a484ff154b9991ec9452  height=0  log2_work=4.0874628  date=2011-02-02 23:16:42

And in stdout:

set addressindex, will reindex. could take a while.
set spentindex, will reindex. could take a while.
finished loading blocks INDXFAIL
OS_loadfile null size.(/home/decker/.komodo/INDXFAIL/komodostate)
komodo_faststateinit retval.-1
fAddressIndex.1/0 fSpentIndex.1/0
nLocalServices 30000005 1, 1
e46e999d6de5ef95427ea51b8242ef7949de22b340b6a484ff154b9991ec9452 hash vs 0f0f0f0000000000000000000000000000000000000000000000000000000000 ht.1 special.0 special2.0 flag.0 notaryid.-1 mod.1 error
04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb6 <- pubkey
04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb6 <- origpubkey
e46e999d6de5ef95427ea51b8242ef7949de22b340b6a484ff154b9991ec9452 failed hash ht.0
komodod: main.cpp:2647: void InvalidChainFound(CBlockIndex*): Assertion `tip' failed.

-testnet flag crashes komodod

Describe the issue

Please provide a general summary of the issue you're experiencing

starting komodod with the -testnet parameter causes it to crash with the line:

komodod: main.cpp:2704: void InvalidChainFound(CBlockIndex*): Assertion `tip' failed.

I think the -testnet flag was initially designed for just launching and testing smartchains without needing 2 nodes/daemons

It might also be causing this following line to be printed when the parameter is used with a plain komodod:

./komodod -testnet                                                                                                                                                                                  
couldnt open.(/home/gcharang/.komodo/.komodo/komodo.conf) will not validate dest notarizations 

Notice: .komodo/.komodo/komodo.conf , it is wierd and looks like a smart chain's conf location had its name replaced by "komodo"

Using

./komodod -ac_name=TEST -ac_supply=100000000 -testnet

Shows the usual output that creates a conf file at the appropriate location and the line with netmagic but still ends with the Assertion tip' failed.` error

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

1. build komodod from master
2. add -testnet to any set of launch parameters that include komodod
3. watch the daemon's output

Expected behaviour

Gracefully shutdown if no valid -ac_ params are given
Launch a new Smart Chain if the -ac_ params are valid

Actual behaviour + errors

Komodo:

./komodod -testnet
couldnt open.(/home/gcharang/.komodo/.komodo/komodo.conf) will not validate dest notarizations
chainparams_commandline called
call komodo_args.(./komodod) NOTARY_PUBKEY.()
initialized  at 1637849952
Komodo  server starting
nMaxConnections 384
finished loading blocks 
processing /home/gcharang/.komodo/komodostate 10206KB, validated.-1
inds.0x7fe039dc1010 validate /home/gcharang/.komodo/komodostate.ind fsize.2415328 datalen.10451142 n.603832 lastfpos.0
/home/gcharang/.komodo/komodostate.ind validated fpos.10451142
took 1 seconds to process /home/gcharang/.komodo/komodostate 10206KB
fAddressIndex.0/0 fSpentIndex.0/0
komodod: main.cpp:2704: void InvalidChainFound(CBlockIndex*): Assertion `tip' failed.

Smart Chain

./komodod -ac_name=TEST -ac_supply=100000000 -testnet
ASSETCHAINS_SUPPLY 100000000
MAX_MONEY 10000006320417438 100000063.20417438
Created (/home/gcharang/.komodo/TEST/TEST.conf)
chainparams_commandline called
>>>>>>>>>> TEST: p2p.12183 rpc.12184 magic.ee32f43e 3996316734 100000000 coins
call komodo_args.(./komodod) NOTARY_PUBKEY.()
initialized TEST at 1637850038
nMaxConnections 384
finished loading blocks TEST
fAddressIndex.0/0 fSpentIndex.0/0
komodod: main.cpp:2704: void InvalidChainFound(CBlockIndex*): Assertion `tip' failed.
Aborted

The version of Komodo you were using:

Komodo Daemon version v0.7.1-beta1-d7edae28b-dirty

cc: @Alrighttt @dimxy

Indexing UTXOs for CCs

Posted by @Alrighttt in #323:

While we're here, we should ask ourselves if we even want "CC addresses". Is there no better way of indexing utxos? The CC address concept is clever, but it comes with plenty of drawbacks.

These addresses are not visually distinguishable from normal addresses. This has caused confusion in the past.

In some cases, they permanently slow down the function of CCs. Eg: Doing for loops against all utxos owned by a CC address.

There is no guarantee that utxos sent to them have been validated. If a CC must iterate over utxos owned by a CC address, the CC must re-validate each utxo before it can be sure the utxo is what it seems to be.

So generally as I understand it the issue is that to interact with a CC you need a way to find it's currently active UTXOs, like an index. So far it's been common to use "marker addresses" which take advantage of the daemon's index of address -> UTXOs (is that right?), but there are a few issues with Alrighttt outlined in his comment:

a. Not distinguishable from regular addresses
b. No detailed control of index, so in some cases you have to loop over UTXOs
c. Indiscriminate, so any transaction sent to the CC address will end up in the index (related to 2)

One possible solution is to create a general state store, which would provide get and set functions that can be used during validation, and just get in the case of RPCs. This, in combination with a new rule in the validation that any transaction with a CC/OP_EVAL output would go through validation, would allow custom logic to be put in place to decide what UTXOs get indexed, and do so in a more efficient manner.

But before going ahead with this, maybe we should examine if there are any potentially more lightweight solutions (protected addresses?), and if there are any other peripheral issues to consider. Do we want a state store? Also, are there any examples of complex or interesting ways that other CCs use marker addresses?

@Alrighttt @dimxy @Mixa84 @jl777

libzcash::PHGRProof (SproutProofVerifier) don't work when libsnark compiled with enabled optimisations on gcc 11.x

A very deeply hidden bug appears only on GCC/G++ 11.x when the internal libsnark is compiled using highly optimized assembly implementations. It is not the default compilation behavior for Komodod from the KomodoPlatform, but it was always (implicitly) the default build behavior for KomodoOcean.

Steps to reproduce:

1. Install Ubuntu 22.04.2 LTS, which have g++ (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0 by default.
2. Clone https://github.com/KomodoPlatform/komodo repo, master branch.
3. Enable assembly optimisations for libsnark build by build daemon with:

CONFIGURE_FLAGS='CXXFLAGS=-DUSE_ASM=1' zcutil/build.sh -j$(nproc)

Preprocessor directive USE_ASM is default for KomodoOcean, as it used in other sources, not libsnark only, and it set from config/bitcoin-config.h by default.

5. Try to sync SUPERNET assetchain or any other chain which have sprout involved transactions:

./src/komodod -ac_name=SUPERNET -ac_supply=816061

7. You will get an error like this:

2023-07-11 15:01:07 receive version message: /MagicBean:0.8.0beta1/: version 170012, blocks=2600141, us=1.1.1.1:53908, peer=13
2023-07-11 15:01:07 Added time data, samples 13, offset +0 (+0 minutes)
2023-07-11 15:01:07 UpdateTip: new best=000114edf65c405771c40e90b8ac5d626a50c8d497b28b0814dbc0863df4d3ea  height=28746  log2_work=29.498648  tx=41966  date=2018-01-02 21:14:21 progress=0.011056  cache=0.6MiB(2766tx)
2023-07-11 15:01:07 UpdateTip: new best=0001a6c23d96b6a2217040bd5c501b1591481ade182994cac84e5ba826cb9101  height=28747  log2_work=29.498712  tx=41967  date=2018-01-02 21:14:50 progress=0.011056  cache=0.6MiB(2767tx)
2023-07-11 15:01:07 UpdateTip: new best=000078d5165eafa590b7fe09cd9d88643b9da07153705c4e768266e20065406a  height=28748  log2_work=29.498776  tx=41969  date=2018-01-02 21:16:27 progress=0.011056  cache=0.6MiB(2769tx)
2023-07-11 15:01:07 ERROR: CheckTransaction(): joinsplit does not verify
2023-07-11 15:01:07 ERROR: CheckBlock: CheckTransaction failed
2023-07-11 15:01:07 Misbehaving: 8.8.8.8:11340 (0 -> 100)
2023-07-11 15:01:07 InvalidChainFound: invalid block=0000179a03788e84f5853240a1255816ace966732cf302c1be0869da931f1885  height=28749  log2_work=29.49884  date=2018-01-02 21:17:27
2023-07-11 15:01:07 InvalidChainFound:  current best=000078d5165eafa590b7fe09cd9d88643b9da07153705c4e768266e20065406a  height=28748  log2_work=29.498776  date=2018-01-02 21:16:27
2023-07-11 15:01:07 ERROR: ConnectTip(): ConnectBlock 0000179a03788e84f5853240a1255816ace966732cf302c1be0869da931f1885 failed
2023-07-11 15:01:07 InvalidChainFound: invalid block=0000179a03788e84f5853240a1255816ace966732cf302c1be0869da931f1885  height=28749  log2_work=29.49884  date=2018-01-02 21:17:27
2023-07-11 15:01:07 InvalidChainFound:  current best=000078d5165eafa590b7fe09cd9d88643b9da07153705c4e768266e20065406a  height=28748  log2_work=29.498776  date=2018-01-02 21:16:27
2023-07-11 15:01:14 receive version message: /MagicBean:0.8.0beta1/: version 170012, blocks=2600141, us=1.1.1.1:52914, peer=14
2023-07-11 15:01:14 Added time data, samples 14, offset +0 (+0 minutes)

Which means that:

bool operator()(const libzcash::PHGRProof& proof) const
    {
// ...
    }

return false on valid transaction, i.e. ProofVerifier::check on perform_verification returns false from r1cs_ppzksnark_online_verifier_strong_IC.

Of course, the chances that somebody will try to use assembly optimisations for libsnark are very low, but the fact is that in KomodoOcean they are used by default. Furthermore, libsnark compiled even with these optimisations has not had any issues when compiled with gcc 10 from Debian 11. However, gcc 11 seems to have introduced some bug in the library. Additionally, we (and it seems ZCash as well) do not have any proper tests to catch this libsnark behavior and that is the point to consider.

'Make clean' does not clean .a lib files

When 'make clean' for the komodo build dir is called static lib .a lib files are not cleaned.
This might lead to unwanted side effects, f.e.:
add temporary debug printing to the code,
then do 'git reset 'to revert back to the state before the debug prints added,
even after 'make clean' you would still get a build with the debug prints

Suggestion is to clean .a files too.

tokel: tokenv2createtokel able to create invalid tx

tokenv2createtokel must check that the "id" field is less than 33554433 as dictated by

komodo/src/serialize.h

Line 45 in 87fb701

static const unsigned int MAX_SIZE = 0x02000000;

and

komodo/src/serialize.h

Line 329 in 87fb701

if (nSizeRet > (uint64_t)MAX_SIZE)

@dimxy

Add notarisation block height to "txnotarizedconfirmed" RPC

Currently the RPC method
komodo-cli -ac_name=DEX txnotarizedconfirmed e01e36d5cf46367fdd0ab28325408e33925df1ad0bf23de0678c0a02e154fbe6
returns

{
  "result": true
}

Could we add to the response with the block in which the notarisation protecting this transaction occured? E.g.

{
  "result": true,
  "notarisation_block":1150136
}

[discussion] dpow2.0 "wishlist"

I am opening this issue as a place to discuss dpow2.0(for lack of a better name). Some of us had begun work on a potential replacement. You can find this here https://github.com/ssadler/zeno . It seems development of this "zeno" will not continue.

I would like to hear feedback on the community's current perception of dpow, what could be better, what needs rethinking, what is worth keeping, etc. I would like this to include not just the technical aspects of dpow, but also NN elections, easy mining, KMD consensus rules regarding notary nodes, long term implications, costs, etc.

No matter how infeasible or unlikely an idea may be, please share it as it may not be as outrageous as it initially seems(like dpow itself 😄). The first post will be my "wishlist". To clarify, it is unlikely all of these ideas will be implemented. It is meant to facilitate discussion. My "wishlist" are my personal views and mine only.

No ARM release for Komodod version 0.7.2

Describe the issue

No ARM release of KMD daemon V0.7.2 available

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

1. Go to https://github.com/KomodoPlatform/komodo/releases/tag/0.7.2
2. scroll down to the released binaries
3. witness there is no ARM release.

Expected behaviour

A new daemon version should be available for all mainstream platforms. Linux on ARM devices is a mainstream platform and will only grow with the rising energy prices.

The version of Komodo you were using:

Run komodod --version to find out
Komodo Daemon version v0.7.1-beta1-3ae44d672-dirty, which needs updating before KMD blockheight 2963330.

Machine specs:

• OS name + version: Ubuntu 20.04.4 LTS aarch64
• CPU: RK3399
• RAM: 4GB
• Disk size: 1TB
• Disk Type (HD/SDD): NVMe
• Linux kernel version (uname -a): Linux rockpi4a 4.4.154-116-rockchip-g86a614bc15b3 #1 SMP Mon Jan 10 12:03:08 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
• Compiler version (gcc -version): N/A

komodo_notaries is not thread safe

calls to komodo_notaries() sometimes update local static arrays without protecting memory. See

komodo/src/komodo_notary.h

Line 127 in 56590a4

didinit[kmd_season-1] = 1;

A suggested fix would be a mutex around the area that checks or modifies the array. A Read/Write mutex may be a good idea here, as it seems this is an often called function, and the data in the arrays are not updated near as much as they are read.