keno/julia-wasm

Running julia on wasm

Try it out!

There's three ways to try out the current state of the wasm port without building anything yourself.

1. An instance of the Web REPL hosted at https://keno.github.io/julia-wasm/website/repl.htm
2. Using a modified version of the Pluto IDE, available at https://pluto-wasm-backend.netlify.app/editor.html
3. Using the iodide IDE plugin (see https://alpha.iodide.io/notebooks/225/ to get started).

Both use a pre-built version. Please note that this is an extremely early alpha and many things are likely (and known) to be broken.

Julia on WASM - Setup instructions

This repo contains various experiments for setting up julia on wasm. It's intended for collaboration and issue tracking before things are working sufficiently to switch to the appropriate upstream repo: There's three scripts in this repo:

• configure_julia_wasm.sh which will setup all the directories
• build_julia_wasm.sh which will build two copies of julia (one natively for cross compiling the system image, one for wasm)
• rebuild_js.sh which will rebuild just the wasm parts and dump it into the website/ directory which is a hacked up copy of https://github.com/vtjnash/JuliaWebRepl.jl

To get started

Pre-conditions

1. nodejs
2. gcc-multilib and g++-multilib

Emscripten SDK

First install the emscripten SDK

# Install emsdk
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install emscripten-main-64bit binaryen-main-64bit
./emsdk activate emscripten-main-64bit binaryen-main-64bit

Upstream LLVM

Then build and install upstream LLVM

# Install upstream LLVM
git clone https://github.com/llvm/llvm-project
mkdir llvm-build
cd llvm-build
cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang;lld" -DCMAKE_BUILD_TYPE=Release ../llvm-project/llvm
ninja

Setup environment

1. Add LLVM_ROOT = '<PATH_TO_LLVM_HERE>/llvm-build/bin' to $HOME/.emscripten
2. Add llvm-build/bin to $PATH

Bootstrapped build

Finally, you're ready to build julia for wasm

# Do this once
./configure-julia-wasm.sh
# this command may fail at the very end of the build process, that's normal
./build-julia-wasm.sh

This command may fail at the very end with an error like the following:

    JULIA build-native/usr/lib/julia/sys-o.a
syntax: incomplete: premature end of inputErrorException("")
ERROR: LoadError: syntax: incomplete: premature end of input
Stacktrace:
 [1] top-level scope at /root/julia-wasm/julia/contrib/generate_precompile.jl:7

This is expected (we're using a 32bit linux build, but telling it it's wasm to generate a compatible system image, so things are a bit confused). Afterwards, build the wasm build using:

 # Do this after you change something on the wasm side
 ./rebuild_js.sh

Afterwards you may set up a webserver using:

emrun --no_browser --port 8888 website/repl.htm &

At the moment Firefox Nightly seems to have the most complete wasm support and seems to be the fastest, so I'd recommend trying that. After starting the server above, just navigate to localhost:8888/repl.htm

Enable WASM GC support integration (Firefox Nightly)

In about:config enable javascript.options.wasm_gc

Enable WASM Reference Types (Chrome Canary)

google-chrome-unstable  --jsflags="--experimental-wasm-anyref"

julia-wasm's Issues

Static compilation to WebAssembly

Ahead-of-time static compilation to WebAssembly would enable smaller downloads to execute Julia code in the browser. We now have a libjulia that compiles nicely to WebAssembly. How to generate code to link to that is uncertain. Some options include:

1. PackageCompiler style--Here, Julia's --output-bc option is used to create LLVM IR with the wasm32-unknown-unknown triple. This approach is currently really slow for native code generation because it recompiles everything starting from base/sysimg.jl. If code generation can be enabled in the wasm version of Julia, this approach could be done in the browser to generate wasm-compatible code. Emscripten would be used for final compilation/linking to WebAssembly. Right now, Emscripten chokes on the addrspaces that Julia generates.

2. CUDAnative style--In this approach, LLVM IR modules are created for Julia code, similar to how @code_llvm works. This works well for simple code, and it should be fast. It doesn't currently work for recursive code. It also doesn't work well for code that calls libjulia functions. foreigncalls are often converted to calls to raw addresses, and it also has the addrspace issue.

3. Charlotte--By directly compiling Julia's IR to WebAssembly, it is possible to customize the compilation. This should be fast and flexible. The downsides of this approach are: no LLVM optimizations and no automatic linking (LLD is supposed to be able to link wasm files).

Missing GC root somewhere

Same code as #10, but with that fixed, we see some sort of memory corruption, most likely a missing GC root:

julia> to_html = x->(buf = IOBuffer(); show(buf, MIME"text/html"(), x); String(take!(buf)))
getfield(Main, Symbol("##5#6"))()
julia> to_html(methods(gcd))
TypeError(func=:getfield, context="", expected=DataType, got=<?#0x1259600::<?#0x1259620::<?#0x91e1a0::<?#0x91e610::Array{Any, (4,)}[
  Symbol("#self#"),
  :result,
  :cached,
  :params]>>>>)

works fine if GC is disabled GC.enable(false).

jscall

The display code in julide is getting a bit unwieldy. We should hook up a jscall intrinsic, hook up the regular display logic and use jscall to do the dom operations/query JS libraries.

build-wasm LLVM lacks llvm-config

I'm seeing the following in build-wasm. This is with the Emscripten that comes with Arch Linux.

[tshort@arch-workstation]: ~/julia-wasm/julia/build-wasm>$ make  julia-ui-release
find: ‘/home/tshort/julia-wasm/julia/build-wasm/usr/share/julia/stdlib/v1.2/*/src’: No such file or directo
ry
/home/tshort/julia-wasm/julia/deps/llvm.mk:539: warning: overriding recipe for target '/home/tshort/julia-w
asm/julia/build-wasm/usr-staging/llvm-6.0.1-0-wasm.tgz'
/home/tshort/julia-wasm/julia/deps/llvm.mk:535: warning: ignoring old recipe for target '/home/tshort/julia
-wasm/julia/build-wasm/usr-staging/llvm-6.0.1-0-wasm.tgz'
which: no llvm-config in (/home/tshort/julia-wasm/julia/build-wasm/usr/tools)
which: no llvm-config in (/home/tshort/julia-wasm/julia/build-wasm/usr/tools)
make[1]: /home/keno/julia/usr/tools/llvm-config: Command not found
    CC build-wasm/src/APInt-C.o
clang++: warning: optimization flag '-falign-functions' is not supported [-Wignored-optimization-argument]
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:20:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:19:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/None.h:23:6: warning: scoped enumerations are
      a C++11 extension [-Wc++11-extensions]
enum class NoneType { None = 1 };
     ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/None.h:24:23: warning: use of enumeration in
      a nested name specifier is a C++11 extension [-Wc++11-extensions]
const NoneType None = NoneType::None;
                      ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:20:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:20:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/AlignOf.h:113:19: warning: deleted
      function definitions are a C++11 extension [-Wc++11-extensions]
  AlignerImpl() = delete;
                  ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/AlignOf.h:140:59: error: expected '(' for
      function-style cast or type construction
                                      T6, T7, T8, T9, T10>),
                                      ~~~~~~~~~~~~~~~~~~~~^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/AlignOf.h:142:61: error: expected class
      name
                                     T6, T7, T8, T9, T10>)> {
                                                            ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:20:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:22:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:53:32: error: a space is
      required between consecutive right angle brackets (use '> >')
struct isPodLike<std::pair<T, U>> {
                               ^~
                               > >
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:65:23: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using UnderlyingT = typename std::remove_reference<T>::type;
                      ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:73:50: warning: 
      'long long' is a C++11 extension [-Wc++11-long-long]
       std::is_convertible<UnderlyingT, unsigned long long>::value);
                                                 ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:78:59: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
struct add_lvalue_reference_if_not_pointer { using type = T &; };
                                                          ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:83:16: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using type = T;
               ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:89:46: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
struct add_const_past_pointer { using type = const T; };
                                             ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:94:16: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using type = const typename std::remove_pointer<T>::type *;
               ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:99:16: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using type = const T &;
               ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/type_traits.h:104:16: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using type = typename add_const_past_pointer<T>::type;
               ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:20:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:32:15: warning: in-class
      initialization of non-static data member is a C++11 extension [-Wc++11-extensions]
  bool hasVal = false;
              ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:35:22: warning: alias declarations
      are a C++11 extension [-Wc++11-extensions]
  using value_type = T;
                     ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:47:14: warning: rvalue references
      are a C++11 extension [-Wc++11-extensions]
  Optional(T &&y) : hasVal(true) { new (storage.buffer) T(std::forward<T>(y)); }
             ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:49:24: warning: rvalue references
      are a C++11 extension [-Wc++11-extensions]
  Optional(Optional<T> &&O) : hasVal(O) {
                       ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:58:25: warning: rvalue references
      are a C++11 extension [-Wc++11-extensions]
  Optional &operator=(T &&y) {
                        ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:68:32: warning: rvalue references
      are a C++11 extension [-Wc++11-extensions]
  Optional &operator=(Optional &&O) {
                               ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:79:21: warning: variadic templates
      are a C++11 extension [-Wc++11-extensions]
  template <typename... ArgTypes> void emplace(ArgTypes &&... Args) {
                    ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:79:57: warning: rvalue references
      are a C++11 extension [-Wc++11-extensions]
  template <typename... ArgTypes> void emplace(ArgTypes &&... Args) {
                                                        ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:136:3: warning: explicit
      conversion functions are a C++11 extension [-Wc++11-extensions]
  explicit operator bool() const { return hasVal; }
  ^~~~~~~~
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:150:3: error: unknown type name
      'constexpr'
  constexpr T getValueOr(U &&value) const LLVM_LVALUE_FUNCTION {
  ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:150:13: error: member 'T' declared
      as a template
  constexpr T getValueOr(U &&value) const LLVM_LVALUE_FUNCTION {
            ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:150:14: error: expected ';' at end
      of declaration list
  constexpr T getValueOr(U &&value) const LLVM_LVALUE_FUNCTION {
             ^
             ;
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Optional.h:171:50: error: a space is required
      between consecutive right angle brackets (use '> >')
template <typename T> struct isPodLike<Optional<T>> {
                                                 ^~
                                                 > >
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:21:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:17:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/iterator_range.h:39:28: warning: rvalue
      references are a C++11 extension [-Wc++11-extensions]
  iterator_range(Container &&c)
                           ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/iterator_range.h:63:65: warning: rvalue
      references are a C++11 extension [-Wc++11-extensions]
iterator_range<decltype(begin(std::declval<T>()))> drop_begin(T &&t, int n) {
                                                                ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:21:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:20:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:18:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/SwapByteOrder.h:92:52: warning: 
      'long long' is a C++11 extension [-Wc++11-long-long]
inline unsigned long long getSwappedBytes(unsigned long long C) {
                                                   ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/SwapByteOrder.h:92:17: warning: 
      'long long' is a C++11 extension [-Wc++11-long-long]
inline unsigned long long getSwappedBytes(unsigned long long C) {
                ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/SwapByteOrder.h:95:48: warning: 
      'long long' is a C++11 extension [-Wc++11-long-long]
inline signed long long getSwappedBytes(signed long long C) {
                                               ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/SwapByteOrder.h:95:15: warning: 
      'long long' is a C++11 extension [-Wc++11-long-long]
inline signed long long getSwappedBytes(signed long long C) {
              ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:21:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:20:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:113:3: warning: 
      _Static_assert is a C11-specific feature [-Wc11-extensions]
  static_assert(std::numeric_limits<T>::is_integer &&
  ^
/usr/lib/emscripten/system/include/libcxx/__config:805:36: note: expanded from macro 'static_assert'
#   define static_assert(__b, __m) _Static_assert(__b, __m)
                                   ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:21:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:20:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:182:3: warning: 
      _Static_assert is a C11-specific feature [-Wc11-extensions]
  static_assert(std::numeric_limits<T>::is_integer &&
  ^
/usr/lib/emscripten/system/include/libcxx/__config:805:36: note: expanded from macro 'static_assert'
#   define static_assert(__b, __m) _Static_assert(__b, __m)
                                   ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:21:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:20:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:205:3: warning: 
      _Static_assert is a C11-specific feature [-Wc11-extensions]
  static_assert(std::is_unsigned<T>::value, "Invalid type!");
  ^
/usr/lib/emscripten/system/include/libcxx/__config:805:36: note: expanded from macro 'static_assert'
#   define static_assert(__b, __m) _Static_assert(__b, __m)
                                   ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:21:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:20:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:276:1: error: unknown type
      name 'constexpr'
constexpr inline uint32_t Hi_32(uint64_t Value) {
^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:276:11: error: expected
      unqualified-id
constexpr inline uint32_t Hi_32(uint64_t Value) {
          ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:281:1: error: unknown type
      name 'constexpr'
constexpr inline uint32_t Lo_32(uint64_t Value) {
^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:281:11: error: expected
      unqualified-id
constexpr inline uint32_t Lo_32(uint64_t Value) {
          ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:286:1: error: unknown type
      name 'constexpr'
constexpr inline uint64_t Make_64(uint32_t High, uint32_t Low) {
^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:286:11: error: expected
      unqualified-id
constexpr inline uint64_t Make_64(uint32_t High, uint32_t Low) {
          ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:291:23: error: unknown type
      name 'constexpr'
template <unsigned N> constexpr inline bool isInt(int64_t x) {
                      ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:291:33: error: expected
      unqualified-id
template <unsigned N> constexpr inline bool isInt(int64_t x) {
                                ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:720:10: error: use of
      undeclared identifier 'alignTo'
  return alignTo(Value, Align) - Value;
         ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:725:23: error: unknown type
      name 'constexpr'
template <unsigned B> constexpr inline int32_t SignExtend32(uint32_t X) {
                      ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/MathExtras.h:725:33: error: expected
      unqualified-id
template <unsigned B> constexpr inline int32_t SignExtend32(uint32_t X) {
                                ^
In file included from /home/tshort/julia-wasm/julia/src/APInt-C.cpp:4:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/ArrayRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/Hashing.h:49:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/Support/Host.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringMap.h:17:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/StringRef.h:13:
In file included from /home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/STLExtras.h:21:
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:75:13: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using U = AlignedCharArrayUnion<T>;
            ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:100:21: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using size_type = size_t;
                    ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:101:27: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using difference_type = ptrdiff_t;
                          ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:102:22: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using value_type = T;
                     ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:103:20: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using iterator = T *;
                   ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:104:26: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using const_iterator = const T *;
                         ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:106:34: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using const_reverse_iterator = std::reverse_iterator<const_iterator>;
                                 ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:107:28: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using reverse_iterator = std::reverse_iterator<iterator>;
                           ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:109:21: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using reference = T &;
                    ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:110:27: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using const_reference = const T &;
                          ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:111:19: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using pointer = T *;
                  ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:112:25: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using const_pointer = const T *;
                        ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:219:20: warning: rvalue
      references are a C++11 extension [-Wc++11-extensions]
  void push_back(T &&Elt) {
                   ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:238:31: error: use of
      undeclared identifier 'NextPowerOf2'
  size_t NewCapacity = size_t(NextPowerOf2(CurCapacity+2));
                              ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:325:22: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using SuperClass = SmallVectorTemplateBase<T, isPodLike<T>::value>;
                     ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:328:20: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using iterator = typename SuperClass::iterator;
                   ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:329:26: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using const_iterator = typename SuperClass::const_iterator;
                         ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:330:21: warning: alias
      declarations are a C++11 extension [-Wc++11-extensions]
  using size_type = typename SuperClass::size_type;
                    ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:339:46: warning: deleted
      function definitions are a C++11 extension [-Wc++11-extensions]
  SmallVectorImpl(const SmallVectorImpl &) = delete;
                                             ^
/home/tshort/julia-wasm/julia/build-wasm/usr/include/llvm/ADT/SmallVector.h:395:13: warning: default
      template arguments for a function template are a C++11 extension [-Wc++11-extensions]
            typename = typename std::enable_if<std::is_convertible<
            ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
fatal error: too many errors emitted, stopping now [-ferror-limit=]
48 warnings and 20 errors generated.

Add status in readme

It would be fantastic if the README on this repo gave some overview on what works, remains to be done, any plans/timeline for upstream integration, and a general sense of the status of all this.

Link in the README does not produce a working julia-wasm

I tried https://keno.github.io/julia-wasm/website/repl.htm. At one point a while ago, it used to give me a working Julia repl. It no longer does.

Dependencies fail to build with LLVM wasm backend

Since for #3 we will likely need to build with the LLVM backend, we should prepare for that. Currently some of our dependencies fail to build (gmp, pcre2) because they have broken configure tests for built-in functions. The action item here is to figure out what the exact issue is and either to fix the configure checks upstream or to patch LLVM to fix this.

conftest.c:93:6: warning: incompatible redeclaration of library function 'memset' [-Wincompatible-library-redeclaration]
char memset ();
     ^
conftest.c:93:6: note: 'memset' is a builtin with type 'void *(void *, int, unsigned long)'
1 warning generated.
LLVM ERROR: Prototypeless function used with conflicting signatures: memset

Segfault when calling `showable`

julia> function f(x); showable(MIME"text/html"(), x); end
typeof(Main.f)()
julia> f(2)
Exception thrown, see JavaScript console

Proper implementation of foreigncall in interpreter

This all currently relies on manually adding special case foreigncall support for all commonly called functions: https://github.com/JuliaLang/julia/blob/kf/wasm3/src/interpreter.c#L547-L1038. It's gotten us this far, but it's a very ugly hack. We should be able to look up the wasm function pointers, compute the wasm signature from the julia signature and just immediately jump to it. The only thing I couldn't figure out here is how to get that function pointer given the name of the function. I tried something like:

    lookup_julia_fptr: function(name) {
        name = Pointer_stringify(name);
        return Module["asm"]['_' + name].name;
    }

but that doesn't work properly (i.e. the number I get back is not the correct function pointer). If somebody can figure out how to take care of that, the rest should be simple.

Ship a compiled version of inference

Things are working well enough now that it's fairly easy to hit case that are not pre-inferred in the system image, and experience inference through the interpreter under wasm is truly a next level latency hog. I think it's time to try and compile the native code part of the sysimage to wasm as well so that we have access to the compiled version of inference.

Re-enable other stdlibs

A bunch of stdlibs are currently disabled in the wasm system image:

https://github.com/JuliaLang/julia/blob/kf/wasm3/base/sysimg.jl#L523-L547

For most of them that's because they depend on WeakKeyDict, which in turn uses ReentrantLock which is disabled because it depends on the libuv atomics support which is disabled.
https://github.com/JuliaLang/julia/blob/a0b7a7610334d0925f126ae20f5ff2d074e3ba69/base/weakkeydict.jl

I think the right fix for now is to have a dummy implementation of ReentrantLock and then re-enable those stdlibs.

Trouble with NamedTuples

julia> to_html = x->(buf = IOBuffer(); show(buf, MIME"text/html"(), x); String(take!(buf)))
getfield(Main, Symbol("##7#8"))()
julia> to_html(methods(gcd))
TypeError(func=:new, context="", expected=NamedTuple{names, T} where T<:Tuple, got=nothing)
error during run:
nothing

I've seen this in a number of instances, but the above is at least one reproducer. I suspect there's a more general problem with NamedTuples here somewhere.

Tasking

I've been thinking about how to implement tasking on top of wasm. Wasm coroutines are a planned feature (https://github.com/WebAssembly/design/blob/master/FutureFeatures.md#coroutines), but no work has been done toward it, as far as I can tell. I've looked at how go implements goroutines and it looks like they're just using explicitly managed stacks in linear memory (but I'll need to look into this further). That's a bit tricky, because you you can't just push a return address onto the linear stack and jump to it. Tail calls (https://github.com/WebAssembly/tail-call/blob/master/proposals/tail-call/Overview.md) may help there, but I'm not sure they're implemented yet. We'd also need an LLVM pass that splits any function at a call boundary and turns it into push return/tail call.

Some sort of memory corruption

julia> println(pi)
π = 3.1415926535897...
nothing
julia> println(pi)
TypeError(func=:typeassert, context="", expected=Array{UInt8, 1}, got=<?#0x769e10::0>)
error during run:
nothing

In native mode I'd call this a missing GC root, but it's hard to know what the problem is. Looking like some sort of printing buffer maybe?

Writing a symbol in iodide REPL

Seems like it is parsed incorrectly.

Emscripten upstream build

Has anyone tried this with the upstream build that uses Clang to generate WebAssembly? I briefly tried with emsdk's "upstream-latest", and compilation failed of build-wasm failed for me.

Dockerfile?

Anyone have a dockerfile they're using for this?

I'd so like to use Julia instead of JavaScript and docker seems the most straightforward way to make a build environment...

Better REPL

Now that the julia-wasm implementation is minimally functional, it would be nice to have a better REPL, ideally re-using most of REPL.jl but with js-specific implementations where necessary.

JS Interop for arrays

@mdboom asked what our plan for array interop is and pointed us to the python implementation at

https://github.com/iodide-project/pyodide/blob/1f978500a02323589bd2c689865a49914fa145a5/src/hiwire.c

since our memory layout is dense, the same approach (taking a typed subarray of the heap buffer) should work and we should be able to re-use the existing data sharing mechanism to work around any GC issues. An alternative design would be to always back arrays by an appropriate TypedArray (would depend on multi-table support in wasm). I suspect the first design is better and simpler for wasm to optimize, but would be worth benchmarking.

Properly hook up error throwing in iodide REPL

Right now causing an error in the iodide REPL leads to a segfault. We should figure out how to report such errors for pretty printing in iodide.

emcc: error: use of undeclared identifier 'FE_INEXACT'

This is a great project and I want to give it a try.
I noticed the following error in ./build_julia_wasm.sh with emcc (version 2.0.17, a543758eed3c9bff9d14c53d22808c2a057a961c))

/home/abarth/src/julia-wasm/julia/src/jlapi.c:447:14: error: use of undeclared identifier 'FE_INEXACT'                                                                
    ret[0] = FE_INEXACT;                                                                                                                                              
             ^                                                                     
/home/abarth/src/julia-wasm/julia/src/jlapi.c:448:14: error: use of undeclared identifier 'FE_UNDERFLOW'                                                              
    ret[1] = FE_UNDERFLOW;                                                                                                                                            
             ^                                                                                                                                                        
/home/abarth/src/julia-wasm/julia/src/jlapi.c:449:14: error: use of undeclared identifier 'FE_OVERFLOW'                                                               
    ret[2] = FE_OVERFLOW;                                                                                                                                             
             ^                                                                                                                                                        
    CC build-wasm/src/jloptions.o                                                                                                                                     
/home/abarth/src/julia-wasm/julia/src/jlapi.c:450:14: error: use of undeclared identifier 'FE_DIVBYZERO'                                                              
    ret[3] = FE_DIVBYZERO;                                                                                                                                            
             ^                                                                                                                                                        
/home/abarth/src/julia-wasm/julia/src/jlapi.c:451:14: error: use of undeclared identifier 'FE_INVALID'                                                                
    ret[4] = FE_INVALID;                                                                                                                                              
             ^                                                                     
2 warnings and 5 errors generated.                                                                                                                                    
2 warnings generated.                               

It seems that these macros have been (intentionally) removed from emscripten:
emscripten-core/emscripten#11087
emscripten-core/emscripten#13678

Shipping julia-wasm with atom-community

I started up the atom-community builds for Atom IDE. I would like to integrate julia-wasm to Atom (which is an Electron app), so people can run web applications using that julia-wasm backend

https://github.com/atom-ide-community/atom/projects/3#card-41196586

What are the steps for doing so? (I prefer shipping prebuilt binaries). I think we need to fix the CI stuff so we can release built binaries. That should be the simplest option.

This is similar to integrating Julia into a browser, so learning the process would be helpful in general.