zilliqa/scilla-rtl

Execute Scilla code compiled by the Scilla -> LLVM compiler.

Scilla Runtime Library

The Scilla Runtime Library provides two main functionalities

1. An entry point to execute Scilla contracts. The contract to be executed must be compiled by the Scilla LLVM compiler, and linked into a shared library object.
2. A collection of functions that implement various common (i.e., not specific to a contract) Scilla operations and enable the compiled binary to interact with the blockchain during execution.

Build and install

Ubuntu:

• sudo apt-get install build-essential clang-13 cmake libboost-dev libboost-test-dev libjsoncpp-dev libboost-filesystem-dev libboost-program-options-dev libsecp256k1-dev
• Use the LLVM apt repository if clang-13 is not in your OS repository.

MacOSX:

• Requires libboost-container-dev in addition to the packages above.

We suggest building ScillaRTL in a directory that is not the source directory.

• $git clone --recurse-submodules https://github.com/Zilliqa/scilla-rtl.git
• $cd scilla-rtl; mkdir build; cd build
• $cmake .. configures the project. Additional (optional) flags:
  • -DCMAKE_INSTALL_PREFIX=/where/to/install/scilla-rtl: To specify an install directory other than the default. The default installation path typically requires root permissions.
  • -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel]: The default build is Debug.
• $make builds the entire project. You can find the built files in bin/ and lib/.
• $make install installs the project.
• $make runtests runs the testsuite. We suggest to provide your installation path as described earlier and not install in a system directory.

Run

While ScillaRTL is intended to be primarily used as a library from the Zilliqa blockchain, we provide two wrapper executables for development, debugging and simply trying it out. expr-runner takes as input, a compiled pure Scilla expression and executes it, by calling a wrapper function scilla_main generated by the compiler. If the expression is printable, the compiler also generates a call to print it. scilla-runner takes a compiled Scilla contract, a message JSON, an initial state for execution and a contract info JSON (obtrained by running scilla-checker with the -contractinfo flag). With these inputs, the transition specified in the message JSON is executed. The output state is printed.

$build/bin/expr-runner -g 1000 testsuite/expr/lit-pair-list-int.ll

$build/bin/scilla-runner -g 10000 --input-contract testsuite/contr/simple-map.ll --message testsuite/contr/simple-map.message_Increment.json --contract-info testsuite/contr/simple-map.contrinfo.json --state testsuite/contr/simple-map.state_00.json --init testsuite/contr/empty_init.json --blockchain testsuite/contr/blockchain_default.json

Developer Notes

All public headers are placed in include. The library implementation resides in libScillaRTL. Executable wrappers are defined in runners. The tests sources are confined to testsuite.

Coding Standards

This project uses the LLVM coding standards.

For convenience, the naming convention is summarized below. All names (with allowed exceptions) must be in camel case

• Types (structs, classes etc), namespaces and filenames begin with a capital letter.
• Function and method names being with a small letter.
• Variable (local, global and class members) names being with a capital letter.
• Functions that serve as Scilla builtins, accessible from the JIT'ed code must start with an _ and be in snake case.

A few other points to note

• Add function comments at declaration points rather than definition points. Definition points can have more details if necessary.
• Include system headers first, then library headers and lastly project headers. Each category separated by a line. clang-format takes care of arranging them in alphabetical order.
• Since this library is designed to be part of an ever running process in the Zilliqa network, do not abort(), exit() or assert(). Always use the provided CREATE_ERROR and CREATE_ERROR_SLOC macros. This throws an exception (with information on where it was thrown from) and can be caught and handled by the blockchain. For assertions, use ASSERT or ASSERT_MSG. These use CREATE_ERROR internally and are defined to be no-op in release builds.

To conform to the coding style and good programming practices, CMake targets clang-format and clang-tidy are provided, which when run as make clang-format and make clang-tidy in the build directory, will auto format all source files.

They can also be manually run from the command line:

• clang-format -style=LLVM -i `find . -name "*.cpp" -o -name "*.h" | xargs
• clang-tidy `find . -name "*.cpp" -o -name "*.h" | xargs` -p build/

Testsuite

The testsuite, built by default (but not installed), is based on boost_unit_framework and can be run from the project root as

build/testsuite/scilla-testsuite -- testsuite_dir

A few command line options are provided below for quick reference. See --help for all options provided by boost. These options must all be provided prior to -- in the command line.

• --list_content: Lists the testsuite hierarchy.
• --run_test : Run specific tests (as listed by --list_content). See details here
• --log_level=all: To enable the full log to be printed.

The testsuite_dir argument following -- is a custom argument that tells the testsuite where to find the tests and their inputs. The custom flag --update-result can be provided to update test results instead of comparing.

For convenience a CMake target runtests has been provided to run the testsuite. This can be executed as make runtests in the build directory.

Expression execution tests

The directory testsuite/expr contains text LLVM-IR files, all generated by expr-compiler. We maintain the following conventions:

• Each test file has the input Scilla expression as a comment at the beginning. Such a file can be generated using the script scripts/gen_expr_test.sh by providing the path to Scilla expression file (.scilexp) in the compiler repository testsuite. (It will implicitly look for the corresponding .gold file already generated by the compiler in the compiler testsuite). A wrapper script scripts/gen_expr_tests.sh is provided to update all expression tests in the testsuite, provided a path to Scilla compiler source directory.
• Each test LLVM-IR file (foo.ll) must have a corresponding foo.ll.result file that contains the expected output on executing the code. The testsuite will match against it.

Contract execution tests

The directory testsuite/contr contains text LLVM-IR files, all generated by scilla-compiler. The directory also contains the supporting JSONs required for executing transitions in these compiled contracts. Each LLVM-IR file foo.ll also has foo.dbg.ll that is a compiled version of the same contract with debuginfo. These LLVM-IR files can be updated from the Scilla LLVM compiler using the scripts/update_contrs.sh. This script updates both the debug and non-debug LLVM-IR modules and also the contract info for that contract.

scilla-rtl's Issues

Check `_sender`'s balance before `accept`ing

Test remote_state_reads_127 is disabled because of this (#45).

Support `_balance`

Zilliqa/scilla-compiler#73

Blockchain Info communication must happen via `ScillaParams`

ScillaContrExec::execMsg takes the current blocknumber as an argument. Instead this should happen via callbacks, similar to how ScillaParams::fetchStateValue etc works, using the newly introduce fetchBlockchainInfo API introduced in Zilliqa/Zilliqa#2757.

Additional reference: Zilliqa/scilla#1056

Dynamic typechecks must be exposed via a builtin

This is needed to support typecast statements.

Zilliqa/scilla-compiler#74

Make the VM thread-safe

• There are global variables whose access needs to be mutexed. Here's a non-exhaustive list.
  • JSON read/writer in Utils.cpp
  • ScillaStdout in SRTL.cpp
• LLVM context and other LLVM objects accessed, as necessary.
• The object code cache manager, both at the memory level and disk level.
• Secp256k1 context has been defined to be a global variable to save time. This isn't thread-safe.

Refactor building of `ScillaJIT` object

On release builds, creating a ScillaJIT object takes ~10ms, which an actual message execution takes < 1ms.

While one solution is to cache ScillaJIT objects, separately created for each popular contract and reuse them, it is better to see if its creation can be refactored so that multiple contracts can use the same ScillaJIT object. Note that all Scilla allocated memory is freed after each message execution / deployment anyway.

The main challenge is that different Scilla modules have same named identifiers in them, so LLJIT will have trouble resolving names when multiple of these object files are loaded.

Verify validity of signed division in wideint

Following #3 , we use wideint from https://github.com/michaeljclark/bignum.

While addition, subtraction and multiplication for 2s compliment signed and unsigned integers are same, division is not. See our OCaml implementation.

Need to verify if the remainder algorithm implemented in wideint is appropriate for Scilla.

See https://en.wikipedia.org/wiki/Modulo_operation#In_programming_languages

Upgrade to C++17

1. boost::any to std::any.
2. boost::optional to std::optional.
3. std::for_each to std::for_each_n where applicable.

Remote load balance of sender must be handled specially

remote_load in EvalUtil.ml subtracts incoming _amount from a balance query for _sender. Do this in SRTL too.

Do not always use `BOOST_REQUIRE` in the testsuite.

BOOST_REQUIRE aborts the test case, and doesn't print useful checkpoints. Use BOOST_CHECK instead.

Migrate expected-to-fail tests for `remote_state_reads`

This is a continuation of #25

Parse unknown assignable types

In #25 , there are tests introduced in which the type specified in the state of a remote contract aren't defined in the contract being tested. However, these types are assignable to the ones in the contract being tested.

For example, the test init_assignable_map_types defines a remote field admin with type T1 : ByStr20 with contract field x : 0x1234567890123456789012345678901234567890.LocalType end. This type fails to be parsed because LocalType isn't known to the VM (as there can be no type descriptor generated for it by the compiler). The requirement for admin is only T : ByStr20 with end as defined in remote_state_reads.scilla. i.e., T1 is assignable to T, and hence this must be parsable and accepted.

Another example. In test _4, state_4.json specifies owners to have type Map (ByStr20 with contract field paused : Bool end) Bool. This type is nowhere defined in the contract and hence no type descriptor exists. It is however assignable to Map (ByStr20 with end) Bool which is what is expected for the remote field owners in the contract.

I suppose the solution is for the type parser to accept an additional parameter "expected type" and skip parsing at points where assignability is already satisfied, so that parsing unknown types won't be an error.

Also see Zilliqa/scilla-compiler#68

Improve dynamic typecheck fail messages

The interpreter classifies dynamic type check fails into four categories evalTCResult in EvalUtil.ml. Do something similar in the runtime library, rather than the current "dynamic typecheck failed" general message.

This will require carrying over a failure message / code, ideally done though std::expected, but that isn't officially there in C++ yet, so maybe an unofficial one ?

Add support for remote state reads

Zilliqa/scilla-compiler#61

Implement missing bits in gas mechanics

In continuation to #12 , there are few things that still need to be implemented:

1. Support for gas_charge elements ListLength, DivCeil and LogOf. The latter two needs code generation in the compiler as well.
2. The interpreter scales the gas limit before beginning execution and later scales it back down. This should be performed in the VM as well (Zilliqa/scilla#877)
3. There are some fixed costs such as contract size + init file size charging during deployment, and message JSON size during transition execution that are missing in the VM.

Implement safe arithmetic for integer arithmetic builtins

The run-time library (SRTL) currently uses boost::multiprecision::cpp_int to represent values of types Int128, Int256, Uint128, Uint256. While this works for unsigned integers, boost uses a sign-magnitude representation for signed integers (rather than 2s complement). Scilla's signed integers however had 2s complement semantics.

This means that while the Int256.min in Scilla is -57896044618658097711785492504343953926634992332820282019728792003956564819968, boost's Int256.min is -115792089237316195423570985008687907853269984665640564039457584007913129639935. This isn't a problem that can be fixed by using a boost type with one less bit (as remarked in the boost documentation itself) because 2s complement allows one extra negative value than the max positive value, but that is not the case with sign-magnitude representation.

There are two ways we can consider solving this:

1. Write wrappers around boost's Int256 (and Int128) to prevent / detect negatives going beyond a value.
2. Write safe-math wrappers around GMP. This can possibly be done by wrapping safe-math around boost's cpp_int too (but with arbitrary sizes).

A test case with wrong output already exists.

Refactor `pow` builtin to work in logarithmic time

Re-implement Zilliqa/scilla#1063 in SRTL. (pow is implemented in SafeInt.cpp).