alexdovzhanyn/alchemyvm

WebAssembly Virtual Machine Built In Elixir

alchemyvm's Issues

Expose an API for interacting directly with a module's memory

It should work similarly to how the browser allows Javascript code to access wasm memory. This will allow more dynamic applications to be built on top of the virtual machine (such as things that interact with a database).

Allow decoded modules to be passed in directly

If we can take a module in directly, rather than only its binary representation, we can skip the decoding step of the VM altogether and go straight to verification / execution. This would allow for caching on the user end and improve performance.

This will involve having some way of decoding a module and having it returned to the end user for storage, and a function that takes in a decoded module and initializes it within the VM

Data section segments should be reflected in memory

Right now we're not doing anything with the values present in the data section. These values should actually initialize the specified location in memory with the given value.

Allow custom sections with a name other than `name`

I've got a custom section in my wasm module that has a name jwt. When I attempt to load this module, I get an error. It appears from the code that your parser will throw an exception upon encountering any custom section other than name. This, I believe, breaks the spec rules allowing anyone to embed any custom section with any name. Can we just get the section added to the module as a binary blob if you don't know how to parse it? Then I can turn it into a string and do the decoding myself later.

This is the error I get:

08:40:34.286 [error] Task #PID<0.403.0> started from #PID<0.398.0> terminating
** (RuntimeError) Parser for custom section jwt not implemented
    (alchemy_vm) lib/decoding/custom_section_parser.ex:20: AlchemyVM.Decoder.CustomSectionParser.parse_section/2
    (alchemy_vm) lib/decoding/custom_section_parser.ex:12: AlchemyVM.Decoder.CustomSectionParser.parse/1
    (elixir) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Function: &:erlang.apply/2
    Args: [#Function<0.83825848/1 in AlchemyVM.Decoder.parallel_decode/1>, [{0, <<3, 106, 119, 116, 101, 121, 74, 48, 101, 88, 65, 105, 79, 105, 74, 113, 100, 51, 81, 105, 76, 67, 74, 104, 98, 71, 99, 105, 79, 105, 74, 108, 90, 68, 73, 49, 78, 84, 69, 53, 73, 110, 48, 46, 101, ...>>}]]
** (EXIT from #PID<0.398.0>) shell process exited with reason: an exception was raised:
    ** (RuntimeError) Parser for custom section jwt not implemented
        (alchemy_vm) lib/decoding/custom_section_parser.ex:20: AlchemyVM.Decoder.CustomSectionParser.parse_section/2
        (alchemy_vm) lib/decoding/custom_section_parser.ex:12: AlchemyVM.Decoder.CustomSectionParser.parse/1
        (elixir) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
        (elixir) lib/task/supervised.ex:35: Task.Supervised.reply/5
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Calculate Gas

We need a way of calculating the 'relative price' of each instruction, and accumulating the total execution cost as a given program runs. We also need an option to specify a 'gas limit', which is the maximum execution cost we're allowed to expend on a given program before we stop. If the 'gas limit' is reached (or exceeded) the vm should stop function execution immediately.

get_global no implemented

WaspVM.load_file(ref, "../AtomVM/src/AtomVM.wasm")
** (RuntimeError) Not implemented: :get_global in init expression
    (wasp_vm) lib/decoding/global_section_parser.ex:58: WaspVM.Decoder.GlobalSectionParser.evaluate_init_expr/2
    (wasp_vm) lib/decoding/global_section_parser.ex:33: WaspVM.Decoder.GlobalSectionParser.parse_entries/2
    (wasp_vm) lib/decoding/global_section_parser.ex:16: WaspVM.Decoder.GlobalSectionParser.parse/1
    (wasp_vm) lib/decoding/decoder.ex:53: WaspVM.Decoder.do_decode/2
    (wasp_vm) lib/wasp_vm.ex:41: WaspVM.load_file/2

I'm getting errors running the tests

Operating system:

uname -a
Linux olafura-GS73-Stealth-8RF 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
No LSB modules are available.
Distributor ID: neon
Description:    KDE neon User Edition 5.14
Release:        18.04
Codename:       bionic

I'm using asdf, here are the versions:

Erlang:
21.0.5
erl --version
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe]

Elixir:

1.7.4
elixir --version
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe]

Elixir 1.7.4 (compiled with Erlang/OTP 21)

.

  1) test 32 bit uint can rem properly (WaspVM.ExecutorTest)
     test/executor_test.exs:106
     Assertion with == failed
     code:  assert WaspVM.execute(pid, "i32__rem_u", [5, 2]) == {:ok, 1}
     left:  {:ok, 2}
     right: {:ok, 1}
     stacktrace:
       test/executor_test.exs:109: (test)



  2) test 64 bit float with sub properly (WaspVM.ExecutorTest)
     test/executor_test.exs:49
     Assertion with == failed
     code:  assert WaspVM.execute(pid, "f64__sub", [4.0, 2.0]) == {:ok, 2.0}
     left:  {:ok, -2.0}
     right: {:ok, 2.0}
     stacktrace:
       test/executor_test.exs:52: (test)

.

  3) test 32 bit int / ns can shl properly (WaspVM.ExecutorTest)
     test/executor_test.exs:186
     Assertion with == failed
     code:  assert Kernel.round(answer) == 4294966496
     left:  4294967296
     right: 4294966496
     stacktrace:
       test/executor_test.exs:194: (test)

.

  4) test 32 bit signed int can divide properly (WaspVM.ExecutorTest)
     test/executor_test.exs:88
     Assertion with == failed
     code:  assert answer == 4294967294
     left:  2147483648.0
     right: 4294967294
     stacktrace:
       test/executor_test.exs:93: (test)

.

  5) test 32 bit integers with sub properly (WaspVM.ExecutorTest)
     test/executor_test.exs:31
     Assertion with == failed
     code:  assert WaspVM.execute(pid, "i32__sub", [4, 2]) == {:ok, 2}
     left:  {:ok, -2}
     right: {:ok, 2}
     stacktrace:
       test/executor_test.exs:34: (test)

..

  6) test 32 bit float with sub properly (WaspVM.ExecutorTest)
     test/executor_test.exs:43
     Assertion with == failed
     code:  assert WaspVM.execute(pid, "f32__sub", [4.0, 2.0]) == {:ok, 2.0}
     left:  {:ok, -2.0}
     right: {:ok, 2.0}
     stacktrace:
       test/executor_test.exs:46: (test)

.

  7) test 32 bit uint can shr properly (WaspVM.ExecutorTest)
     test/executor_test.exs:231
     Assertion with == failed
     code:  assert Kernel.round(answer) == 536870899
     left:  2147483661
     right: 536870899
     stacktrace:
       test/executor_test.exs:240: (test)



  8) test 64 bit uint can rem properly (WaspVM.ExecutorTest)
     test/executor_test.exs:112
     Assertion with == failed
     code:  assert WaspVM.execute(pid, "i64__rem_u", [5, 2]) == {:ok, 1}
     left:  {:ok, 2}
     right: {:ok, 1}
     stacktrace:
       test/executor_test.exs:115: (test)



  9) test 64 bit int can shl properly (WaspVM.ExecutorTest)
     test/executor_test.exs:197
     Assertion with == failed
     code:  assert round(answer) == 18446744073709550816
     left:  18446744073709551616
     right: 18446744073709550816
     stacktrace:
       test/executor_test.exs:206: (test)

..

 10) test 32 bit unsgined int can divide properly (WaspVM.ExecutorTest)
     test/executor_test.exs:81
     Assertion with == failed
     code:  assert result == {:ok, 2}
     left:  {:ok, 0}
     right: {:ok, 2}
     stacktrace:
       test/executor_test.exs:85: (test)



 11) test 32 bit float can divide properly (WaspVM.ExecutorTest)
     test/executor_test.exs:96
     Assertion with == failed
     code:  assert result == {:ok, 2.0}
     left:  {:ok, 0.5}
     right: {:ok, 2.0}
     stacktrace:
       test/executor_test.exs:100: (test)

..

 12) test 32 bit sint can shr properly (WaspVM.ExecutorTest)
     test/executor_test.exs:210
     Assertion with == failed
     code:  assert Bitwise.band(result, 4294967295) == 4294967283
     left:  0
     right: 4294967283
     stacktrace:
       test/executor_test.exs:216: (test)

..

 13) test 64 bit sint can shr properly (WaspVM.ExecutorTest)
     test/executor_test.exs:219
     Assertion with == failed
     code:  assert Bitwise.band(result, 18446744073709551615) == 18446744073709551603
     left:  0
     right: 18446744073709551603
     stacktrace:
       test/executor_test.exs:226: (test)

..

 14) test 32 bit sint can rem properly (WaspVM.ExecutorTest)
     test/executor_test.exs:118
     Assertion with == failed
     code:  assert WaspVM.execute(pid, "i32__rem_s", [-5, 2]) == {:ok, 4294967295}
     left:  {:ok, 4294967294.0}
     right: {:ok, 4294967295}
     stacktrace:
       test/executor_test.exs:121: (test)

...

 15) test 64 bit integers with sub properly (WaspVM.ExecutorTest)
     test/executor_test.exs:37
     Assertion with == failed
     code:  assert WaspVM.execute(pid, "i64__sub", [4, 2]) == {:ok, 2}
     left:  {:ok, -2}
     right: {:ok, 2}
     stacktrace:
       test/executor_test.exs:40: (test)

.........

Finished in 0.3 seconds
42 tests, 15 failures

Randomized with seed 22201

Element Section Parser

Element section is not implemented yet, but a file for it exists in decoding/element_section_parser.ex

Indirect function calls

Allow functions outside of a given module to be called from within -- via imports. Function definitions will need to be provided by the host environment when the module is initialized.

Module initialization should throw an error if there is not a value passed in for each import within the module.

Implement table section parser

It's currently partially implemented in decoding/table_section_parser.ex

Custom Section Parser

Wasm Spec specifies support for custom sections (sections with id 0)

Module Verification

After transforming a module from binary to our internal instruction format, we need to validate that a given module is valid, according to the validation logic specified in the Wasm Validation Section and other places in the spec

Handle Traps

There are a few situations where we need to trap in the executor, currently we're just crashing, lets create a dedicated function that'll gracefully handle traps.

Parallel Decoding

Wasm binary format is structured in such a way that it should be easy to decode its sections independent of each other. We can make use of Elixir's cheap concurrency to decode each section of a module in parallel.

We do an 'initial' decoding passthrough where each of the modules sections are split into its respective parts as binaries. After this, we decode each section one by one. It should be a rather simple optimization to spin up a process for each of the sections which decodes them, and then merges them back into the module.

Parse current_memory instruction

Trying the language poetry, I found WaspVM doesn't currently parse current_memory, which is not mentioned at the specification but seems to be valid on firefox, chrome and nodejs. Related issue on the spec: WebAssembly/design#904

Minimal poetry code which uses current_memory

export_memory "memory"

export "init" init
  return true

WAT file

---

Traceback of the error

[error][ ] Task #PID<0.1283.0> started from #PID<0.1206.0> terminating
** (RuntimeError) Couldn't parse instruction for current_memory
    (wasp_vm) lib/decoding/instruction_parser.ex:193: WaspVM.Decoder.InstructionParser.parse_instruction/2
    (wasp_vm) lib/decoding/code_section_parser.ex:70: WaspVM.Decoder.CodeSectionParser.parse_bytecode/2
    (wasp_vm) lib/decoding/code_section_parser.ex:36: WaspVM.Decoder.CodeSectionParser.parse_bodies/2
    (wasp_vm) lib/decoding/code_section_parser.ex:12: WaspVM.Decoder.CodeSectionParser.parse/1
    (elixir) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Function: &:erlang.apply/2
    Args: [#Function<0.26794313/1 in WaspVM.Decoder.parallel_decode/1>, [{10, <<109, 23, 1, 2, 127, 16, 46, 2, 64, 65, 5, 33, 0, 12, 0, 65, 1, 33, 1, 11, 32, 0, 16, 47, 11, 8, 0, 32, 0, 16, 52, 16, 99, 11, 8, 0, 32, 0, 16, 54, 16, 99, 11, 8, 0, ...>>}]]
** (EXIT from #PID<0.1206.0>) shell process exited with reason: an exception was raised:
    ** (RuntimeError) Couldn't parse instruction for current_memory
        (wasp_vm) lib/decoding/instruction_parser.ex:193: WaspVM.Decoder.InstructionParser.parse_instruction/2
        (wasp_vm) lib/decoding/code_section_parser.ex:70: WaspVM.Decoder.CodeSectionParser.parse_bytecode/2
        (wasp_vm) lib/decoding/code_section_parser.ex:36: WaspVM.Decoder.CodeSectionParser.parse_bodies/2
        (wasp_vm) lib/decoding/code_section_parser.ex:12: WaspVM.Decoder.CodeSectionParser.parse/1
        (elixir) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
        (elixir) lib/task/supervised.ex:35: Task.Supervised.reply/5
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Implement Table Initialization

We need a data structure in the module instance to represent tables as defined in the table section of any given WebAssembly module. This initialization will be based on the elements provided in the element section of the module. An element is to a table what a data section is to memory.

This initialization needs to happen in module_instance.ex, where all the other module attributes are initialized.

Issue #4 is a blocker for this issue.

Data Section Parser

We need to parse section 11 (data section) if a given module has it

Add debugger binding support