crypto-com/thaler

Thaler Experimental Network; For Crypto.org Chain: github.com/crypto-org-chain/chain-main

DEVELOPMENT MOVED TO: https://github.com/crypto-org-chain/chain-main

Table of Contents

1. Description
2. Contributing
3. License
4. Documentation
5. Build
   1. Docker image
   2. Makefile
   3. Linux native (Ubuntu 18.04)
   4. Build mock mode on non-sgx platform (e.g. mac)
6. Start a Local Full Node
7. Send your First Transaction
8. Testing
9. Useful Links

1. Description

This repository contains the Thaler Experimental Network code (pre-pivoted Crypto.com Chain). The current repository consists of the following Rust sub-crates:

• chain-abci: the Tendermint ABCI application that currently does the transaction validation etc.
• chain-core: the library that contains the main type definitions and any utility code (such as serialization), so that it can be used in chain-abci and other applications.
• chain-storage: storage related logic used by chain-abci.
• chain-tx-filtering: Library that captures the fuctionality related to block-level public view key-based transaction filtering.
• chain-tx-validation: Library with functions that verify, given current chain state's data, if a transaction is valid.
• test-common: Common code shared by unit tests.
• dev-utils: currently a minimal development tool for generating genesis.json
• client-[common|network|core|cli|rpc]: Client backend implementation for transaction creation and wallet management. Follow these links for more details:
  • client-common
  • client-core
  • client-rpc
• chain-tx-enclave/: enclaves and enclave wrapper apps.
• enclave-protocol: Requests and responses exchanges over ZMQ between chain-abci app.
• cro-clib: c API library.

2. Contributing

Please abide by the Code of Conduct in all interactions, and the contributing guidelines when submitting code.

3. License

Apache 2.0

4. Documentation

Technical documentation can be found in this Github repository (you can read it in this hosted version.

5. Build full node

1. Docker image

$ docker build -t crypto-chain:latest .

Docker build arguments:

• SGX_MODE:
  • HW: SGX hardware mode, default.
  • SW: SGX software simulation mode.
• NETWORK_ID: Network HEX Id of Tendermint, default: AB.
• BUILD_PROFILE:
  • debug: debug mode.
  • release: release mode, default.
• BUILD_MODE:
  • sgx: default.
  • mock: A simulation mode only for development on non-sgx platform, don't use in production.

2. Makefile

$ make build

It builds in docker container, the result binaries reside in local directory, it runs something like:

$ docker run --rm -v `pwd`:/chain cryptocom/chain:latest run_build_scripts

The result binary is built for the docker container environment, may not runnable locally.

The makefile supports other commands too:

$ make help
...
SUBCOMMAND:
	prepare                prepare the environment
	image                  build the docker image
	build                  just build the chain and enclave binaery in docker
	run-sgx                docker run sgx-validation and a sgx-query container
	run-chain              docker run chain-abci, tendermint and client-rpc container
	stop-all               docker stop all the container
	start-all              docker start all the container
	restart-all            docker restart all the container
	rm-all                 remove all the docker container
	clean                  clean all the temporary files while compiling
	clean-data             remove all the data in data_path

3. Linux native (Ubuntu 18.04)

Prerequisite:

• intel sgx sdk (Set environment variable SGX_SDK to the sdk directory)
• rust toolchain nightly-2019-11-25 (you can install with rustup)

$ apt-get install -y \
    cmake \
    libgflags-dev \
    libzmq3-dev \
    pkg-config \
    clang
$ ./docker/build.sh

All the executables and signed enclave libraries will reside in ./target/debug.

Environment variables mentioned in the docker image building section also apply here.

4. Develop with mock mode on non-sgx platform (e.g. mac)

TODO

6. Start a Local Full Node

Please follow the instruction to deploy a local full node.

7. Send Your First Transaction

Kindly refer to this instruction to perform transactions between addresses.

8. Testing

You can run the unit tests and integration tests with drone-cli on sgx platform:

$ cat > .drone.secret << EOF
SPID=<SPID>
IAS_API_KEY=<IAS_API_KEY>
EOF
$ drone exec --trusted \
    --include build \
    --include unit-tests \
    --include integration-tests \
    --include multinode-tests

Kindly refer to Prepare SPID & KEY to obtain the values of SPID and IAS_API_KEY.

---

9. Useful links

• Project Website
• Technical Documentation
• Community chatrooms (non-technical): Discord Telegram
• Developer community chatroom (technical):

thaler's Issues

(CRO-473) Problem: Tx-proof not found for multi-sig demo on slow machine

Debug Notes from @lezzokafka :

Forget to set it to zero-fee;

Tendermint/RPC web-socket changes

SLOW network/machine

Suggested solution:

Downgrading Calvin Lau Carson Yan Mac

In case of .tx proof not found, re-try.

Sample wallet confirms the TX is on-chain with tendermint beforehand.

(CRO-288) Problem: no application versioning

Currently, we didn’t have any official “release” – once there is (I guess with the testnet), it'll be good to follow some versioning convention for crates – so I guess that would SemVer + https://rfc.zeromq.org/spec:42/C4/#26-evolution-of-public-contracts

All Public Contracts (APIs or protocols) SHALL be documented.

All Public Contracts SHOULD have space for extensibility and experimentation.

A patch that modifies a stable Public Contract SHOULD not break existing applications unless there is overriding consensus on the value of doing this.

A patch that introduces new features SHOULD do so using new names (a new contract).

New contracts SHOULD be marked as "draft" until they are stable and used by real users.

Old contracts SHOULD be deprecated in a systematic fashion by marking them as "deprecated" and replacing them with new contracts as needed.

When sufficient time has passed, old deprecated contracts SHOULD be removed.

Old names SHALL NOT be reused by new contracts.

On the blockchain side, we’ll need to “fix” the binary serialisation / deserialisation for backwards-forwards compatibility (see the linked issues) + have some “global” application version number – in Tendermint: https://tendermint.com/docs/spec/abci/abci.html#info AppVersion is u64 and is included in every BlockHeader… so there needs to be a consensus on this number (if 50% of validators run old version and the other 50% the new, there won’t be a consensus – if 67%+ of validators run the new version, the other ones would be flagged as byzantine)

@all-involved-developers, if you have any more thoughts on this, feel free to comment here

Problem: integration-tests/start-local.sh is not working with mock features

Start-local.sh comes with cargo run, no features flag for mock version

(CRO-464) Problem: no dynamic creation of docker-compose.yml

except for bash scripts, make other scripts dynamic using python3
produce docker-compose.yml on the fly, perform testing

use environment variables to communicate between steps

Binary Release

This epic groups issues related to making it possible to have the first binary release of the ABCI node code and client.

This mainly revolves around API and binary format stabilization.

(CRO-246) massive pubkey test in Merkletree

make thousands of pubkey
and merkletree works ok with maintaning good performance

Problem: no UTXO set commitments

currently, the utxo set is maintained in the full node's internal state as this tx_meta column of bitvectors.

it'll be good to have commitments to UTXO set in the application state / app hash:

1. light client could check/verify if their utxos are a part of the latest utxo set
2. may be useful for full node data bootstrapping -- node will just need to verify the validator set, compute the "diff" between its last stored UTXO set and the latest one, and fetch the needed (i.e. part of the latest UTXO set that are not in the last stored one) transactions from the remote node.

could potentially use something like https://dci.mit.edu/utreexo

(CRO-292) Review maximum transaction sizes once transaction format is stabilised

Problem: signer app doesn't allow specifying timelocks

(CRO-151) client GUI binary release (Linux, MacOS, Windows?)

(CRO-645) Problem: no stress test for chain

write python script, which does some stress test to make bad txs
consider it as a part of integration test

1. very abnormal txs
2. very big amount of txs

check that chain doesn't crash, and bloom-filter works well.

(CRO-43) Upgrading/release mechanism: developer sign-offs, ...

As the binary code will be fixed for TEE remote attestations, there will also need to be a mechanism for upgrades / release procedure. It should follow the Update Framework procedures: https://en.wikipedia.org/wiki/The_Update_Framework_(TUF)But in a more decentralised fashion with skipchains: https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/nikitin

It'll go something like this:1. There'll be a set of developer keys and responsibilities recorded in the skipchain structure(so that they could be updated: revoked, rotated, ...).2. There needs to be a deterministic build procedure: e.g. using Nix and/or Bazel... https://en.wikipedia.org/wiki/Reproducible_builds3. Upon a new release, a set of developers will co-sign the corresponding build's artefacts.4. Special TX will be broadcasted in the network: it will include the release artefact details, signature, a grace period expiration time... 5. Announcements will be made in different off-chain channels + software will display upgrade warnings to admins.6. After the grace period, connections from old versions will be rejected etc.

(CRO-134) TEE-enforced privacy

This epic groups issues related to the transaction privacy that’s achieved by encrypting parts of its data, so that it could only be validated inside SGX enclaves

(CRO-249) refactoring of merkletree to static array

because merkletree is full binary tree.
we can use static array for storing all nodes,and compute roothash recursively.

also it's flattened storage, serialization or deserialization can be simpler.

(CRO-644) Problem: no one-line installation for chain

for example)curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

present one line installation

1. download tendermint
2. download pre-compiled abci, rpc, cli
3. make standard install location as default (.cro or .cro_chain)

(CRO-550) Problem: ambiguous error message

when account does not exist
Error: Deserialization error: Unable to deserialize abci_query from JSON-RPC response for params: [String("account"), String("41c0294b79975f70c8618be33f39388941f76521"), Null, Null]

change to
account does not exist

Problem: client keeps re-deriving hardened key

Instead, it should use hmac-ed keys, as it was originally intended: #723 (review)

A temporary patch for the slow client execution may be submitted if this fix blocks other work.

Staking

(CRO-163) Backwards/forwards on-chain data binary compatibility

Some thoughts: https://docs.corda.net/serialization-enum-evolution.html

(CRO-16) Settlement agents: external cryptocurrency wallet integration + SPV clients...

(CRO-148) periodic enclave-shared encryption key generation

Node whitelist

(CRO-241) multi-signature tx stress test

make fake multisig txswhich contains bogus multi sig txs

make sure those txs are dropped in abci server.

(CRO-17) TEE: secret + ledger data storage, remote attestation, TX validation via TEE

(CRO-660) Problem: no support for batch operations

Not sure if useful – e.g. for sweeping / withdrawals… @calvinaco may comment

(CRO-673) Problem: storage coupled with state machine logic

step 1 of ADR-001 (not pending any decision):

Problem: rust-secp256k1 fork diverged from upstream

when SchnorrSig + MuSig is merged

bitcoin-core/secp256k1#558

bitcoin-core/secp256k1#588

bitcoin-core/secp256k1#572

bitcoin-core/secp256k1#589

bitcoin-core/secp256k1#590

Problem: signer shows wrong addresses

It uses the default .serialize, which in that fork of secp256k1 is the compressed representation.
According to the Ethereum specs, the keccak input / the public key must be uncompressed and 64 bytes long or 65 with the constant 0x04 prefix (and take off the first byte in that case). serialize_uncompressed should give the 65-byte version: https://github.com/crypto-com/rust-secp256k1-zkp/blob/master/src/constants.rs#L29

Perhaps, it'll be better to reuse RedeemAddress from chain-core rather than duplicate this logic in signer-core.

(CRO-19) TEE p2p payment channels and chains

in the style of https://www.teechain.network

(CRO-652) Problem: no clear command for tx-validation

add the command to clear /enclave-storage

example)
tx-validation-app clear
or
tx-validation-app unsafe_reset_all

(CRO-677) Problem: The result of the list address API is not ordered by creation order

The address list returned by the client should be ordered stable, e.g. by creation order.

Python integration tests use address list API to get the bonded/unbonded stacking address, it’s not stable with current API.

Cross-currency

Consensus

As the core of consensus is provided by Tendermint, this epic mainly groups tasks related to making rust-ABCI more robust: https://github.com/tendermint/rust-abci/issues

Problem: no reproducible builds

https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/rust.section.md

https://www.hadean.com/blog/managing-rust-dependencies-with-nix-part-i

Escrow (TO POPULATE)

(CRO-651) Mocking transaction logic in `BlockGenerator`

To use BlockGenerator for more unit testing, we need it to be able to add transactions in blocks, which means it needs to support the whole transaction logic and state transition logic.

(CRO-245) enhancement in combined_hash of Merkletree

instead of copying, we can feed arrays twice,get the same result.

Problem: no need for enclave start-up in Travis integration test

https://github.com/crypto-com/chain/pull/729/files

as it's using mock-validation, there's no need to start up the tx-validation enclave -- may be irrelevant after https://github.com/crypto-com/chain/pull/738/files

(CRO-515) Problem: chain-abci stores validator metadata

chain-abci currently stores a lot of metadata related to validators which is unnecessary because tendermint internally maintains validator set on its own and passes metadata in ABCI Requests whenever needed. The only validator information which we may need to store is:

staking_account_address

liveness and slashing_schedule

(CRO-18) TEE data oracle for benchmark exchange rates in payment agents

(CRO-46) TEE Data Migration / Bootstrap

As data on chain is encrypted, the new node's enclave needs to contact existing nodes to get it.

Full nodes need a way to get data from validators.

When one upgrades a server, load-balances, etc.

"Data migration typically requires a back-end server that verifies the identity of the enclave on the old system and the enclave on the new system, and facilitates the key exchange between the two systems to share the data. Regardless of the specific method that an ISV uses to migrate data, the seal key should not be shared outside an enclave because it could compromise the entire platform."

(CRO-281) Problem: No easy to use API for creating multi-sig transactions

Current API for creating multi-sig transactions is not intuitive and is very hard to use in a real-world scenario. Abstract current API in an easy to use and intuitive design.

(CRO-31) rust-tsp (ABCI) via unix sockets instead of TCP

Performance Monitoring

(CRO-261) Problem: no "init" for existing networks

This issue is what @leejw51crypto had originally in mind when preparing the “developer utility” – as @tomtau mentioned on Slack:

one needs genesis.json

some node network information (address / node_key + IP/HOSTNAME:PORT) to connect to

I think for the first one, it’s easy – just bundle this file in a tarball with binaries and have a bash script that would copy it over…

for the second one, I guess there may be some extra steps to be automated – depends on the pending enclave.

(CRO-340) Problem: No details for new staking address in query response

RPC Error is returned from Tendermint RPC chain_abci(account) call without further details

(CRO-211) Problem: Support AMD TEE tech

This is more of a Wishlist item. AMD has its own SGX-equivalent technology (AMD SME, SVE, PSP), but it has some difference: https://caslab.csl.yale.edu/workshops/hasp2018/HASP18_a9-mofrad_slides.pdf

Currently, the main blockers / problems are:

No Rust SDK (but seems that’s not required?)

No enclave/app-level remote attestation protocol – we’d need to pretty much prepare the whole Operating System and attest on that level via (f)TPM

No enclave memory isolation: SGX isolates enclave from OS and other enclaves, AMD SEV only isolates OS from hypervisors and other OS (if virtual)

Overall, AMD TEE technology has a different trust model than SGX – overall, AMD would require us to prepare the whole OS / hypervisor stack… e.g. based on seL4: http://sel4.systems

---

similar issues are with Arm TrustZone

(CRO-239) client index stress test

make stress test for merchant such as exchange

make many wallets (thousands)
also many utxos (thousands )
write harsh test aging

make sure that client index sync correctly
also make sure that abci server works correctly

(CRO-363) Problem: ABCI stores witness information which can safely be deleted

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

    Updating crates.io index
    Updating git repository `https://github.com/crypto-com/rust-secp256k1-zkp.git`
error: failed to load source for a dependency on `secp256k1zkp`

Caused by:
  Unable to update https://github.com/crypto-com/rust-secp256k1-zkp.git?rev=a67ac07f98e3a9bdd10ff2822ae0340b7962fd2c#a67ac07f

Caused by:
  failed to update submodule `depend/secp256k1`

Caused by:
  object not found - no match for id (1c830b4c9ac30aa32e246ea3ab8ef9ef99acf664); class=Odb (9); code=NotFound (-3)

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.