truebitfoundation/ocaml-offchain

Fork of WebAssembly reference interpreter with support for generating proofs needed for blockchain verification

ocaml-offchain's Issues

Repo License

We need to license the repo properly.
The original repo's license for the interpreter and test folder are Apache V 2.0. i guess it should be fine to relicense it to MIT since its Apache. any thoughts?

Filesystem: output instructions

Add instructions for output. Should be quite similar to input instructions. But there might have to be an instruction that creates a new file with a given size.

Filesystem: write to a file

Research Merkle Tree efficiency

Research different merkle tree's for best performance

System calls: simple stubs

The following system calls just need simple stubs. They could be handled using the generic system calls, but it will be more efficient to add stubs.
29. Pause:nothing to implement
34. Nice: nothing to implement
35. Sync: nothing to implement
42. Pipe: nothing to implement
51. Acct: nothing to implement, but what is it?
66. Setsid: nothing to implement
75. Setrlimit: nothing to implement
96. Getpriority: nothing to implement
97. Setpriority: nothing to implement
104. Setitimer: nothing to implement
114. Wait4: nothing to implement
118. Fsync: nothing to implement?
121. setdomainname: nothing to implement
125. Mprotect: nothing to implement
148. Fdatasync: Nothing
150. Mlock: nothing
151. Munlock: nothing
152. Mlockall: nothing
153. Munlockall: nothing
163. Mremap: nothing
199, 200, 201, 202: getuid32, getgid32, geteuid32, getgid32: nothing
203, 204, 213, 214. Setreuid32, setregid32, setuid32, getuid32: nothing
218. Mincore: nothing
219. Madvise: nothing
265. Clock_nanosleep: nothing
272. Fadvise64: nothing
299. Futimesat: nothing
303. Linkat: nothing
308. Pselect: nothing
331. Pipe2: nothing

How to return 64 bit values?

wasdk generated something with setTempRet0, hopefully this won't have to be implemented.

Filesystem: read from file

GPU hashing

This would speed up hashing capabilities

Add documentation explaining how the code is pre-processed/initialized

Implement emscripten memcpy

Implement emscripten_memcpy_big

Skipped syscalls

These are the syscalls that should not be implemented, the output from recorded system calls can be used instead.
20. Getpid: skip
55. Setpgid: skip
64. Getpid: skip
65. Getpgrp: skip
77. Getrusage: skip
102. Socketcall: skip?
122. Uname: skip?
132. Getpgid: skip
147. Getsid: skip
191. Ugetrlimit: skip
195. Stat64: skip?
196. Lstat64: skip?
197. Fstat64: skip?
205. Getgroups32: skip
208, 210. Setresuid32, setresguid32: skip
209, 211. Getresuid, getresgid: skip
268. Statfs64: skip?
269. Fstatfs64: skip?
300. Fstatat: skip?
340. Prlimit64: skip
320. Utimensat: skip

But for example in stat, we should perhaps check that the file exists, and has correct size.

Initialization and initial state

Currently the specification of the task is the hash of the initial state. Because of this, when a task is posted as wasm file, it has to be associated with an initial state hash. Then the solvers and verifiers can check if the wasm file and the initial state match.
There is a problem if the wasm file and the initial state do not match. The whole initial state could be posted instead, but it might be several times larger than the wasm file (if decoding the instructions is not implemented in solidity).
Solution alternatives:

1. Implement preprocessing in solidity
2. Implement preprocessor as wasm module

The second alternative is easier, for example the ocaml runtime can be compiled into wasm: https://github.com/sebmarkbage/ocamlrun-wasm

Loading WASM files

We have to be able to load these binary files, and then run them, for example main function or something.

Emscripten and input handling

Find out how the special function for input can be called from C.

Emscripten command-line arguments

Command line arguments should be allocated with malloc, initialized, and then passed to main as arguments.

Basic input and output

Some of these already have initial implementations
3. Read
4. Write
5. Open
6. Close
41. Dup: just open with new descriptor
63. Dup2: what does it do?
140. Llseek
145. Readv
146. Writev
180. Pread64
181. Pwrite64
295. Openat
330. Dup3
333. Preadv
334. Pwrited

Implement Parallel Computations

Currently only one computation can be run on VM at a time

File permissions

These can probably be skipped for now
15. Chmod. Should we store file owners etc? At least not in input block, perhaps in a configuration file
94. Fchmod
198. Lchown
207. Fchown32
212. Chown32
298. Chownat
306. Fchmodat

JIT

This would increase performance

Filesystem: open file

Add RPC Server port to Dockerfile

After RPC Server is implemented, we can open up a port on the docker container to make installation/use a breeze

Compiler tool-chain

Find out how to compile something from C or Rust to webasm, and then test if it works with the interpreter.

refactor merge.ml and addglobals.ml

There must be a cleaner way to implement them.

What do these system calls do?

14. mknod . what is this?
15. Access: what is this?
16. Ioctl
17. umask
18. Readlink: ???
19. Msync: ???
20. Fcntl64
21. Mknodat
22. Faccessat
23. Fallocate

Exit system calls

Make sure that it is implemented correctly.

Generic external calls

Link in the generic external calls and make them work. Also in the file system calls, we have to skip the external calls.

Adding initialized memory

Add some memory elements. Perhaps can be done at the same time as configuring globals.
This is needed because the emscripten JS runtime does some memory initialization.

Polling

Hopefully can be skipped
142. Newselect: do we have to implement something?
168. Poll: do we have to implement?

Invoking modules multiple times

Some test cases need to invoke the modules many times. Currently this is not supported.

Error when building Wasm on OSX

i went through similar instructions on High Sierra:

brew install opam ocaml
opam init -y
eval $(opam config env)
opam install cryptokit yojson

Then encountered this error when executing make to build wasm:

# Error: This expression has type bytes but an expression was expected of type
#          string

Windows CI

Do we wanna do a windows CI for the repo as well?

External functions for handling task input

Figure out of to add external functions that can be used to access the data posted from blockchain.

Managing files and directories

Some of these might be easy to implement, but directory handling might be complex.
9. Link (not implemented?)
10. Unlink (remove files?)
12. Chdir, perhaps need something for keeping track of current directory, it will be similar to command line arguments
38. Rename
39. Mkdir: how to implement directories?
40. Rmdir
83. Symlink: how to handle?
133. Fchdir
183. Getcwd
220. Getdents
296. Mkdirat
301. Unlinkat
302. Renameat
304. Symlinkat

Filesystem: importing filesystem.c

wasdk seems to compile filesystem.c to a simple wasm file. This file should be merged to the actual file to be executed, also some linking has to be done when importing builtins.

Filesystem: write the file system from memory to storage

When the program exits, the file system should be output to storage.

The interpreter should throw better error messages

I unsuccessfully tried running a .wasm file using the interpreter. The error messages could be more helpful.

I then tried running the .js file using our emscripten-module-wrapper and got more cryptic error messages.

Turns out the latter was due to me having an older node version.

Change breaks to static jumps

When initializing, the breaks could be changed into static jumps / stack adjustments.

Add CI

Add automated testing.

Always start from empty initial state

Instead of having initialized memory and tables, just have some instructions that will initialize the state.

Importing globals

When setting up globals table, handle importing globals.

Make this a RPC Server

Had the idea of turning this into a JSON RPC Server like Parity or Geth.
Then we write software that can act as a client.

Benefits:
Easier installation for other projects that depend on this (webasm-solidity, coindrop, etc)
Allow us to write client in any language(s) we want
Easier to document the API

Truncating files

Probably these too have to be implemented:
193. Truncate64
194. Ftruncate64

Memory mapping files

91. munmap: how to handle memmapping?
92. Mmap2

VM configuration

Get the VM memory size etc. from command line or a configuration file instead of hard coded parameter.

Multiple modules

Linking imported functions has to be implemented properly.

Emscripten: find out how the filesystem works

There seems to be some kind of .data file.
Probably what needs to be done is that there are builtin functions for keeping track of file descriptors, and instructions that can be used to read and write to some kind of data block.