ocaml-multicore/ocaml-multicore

Multicore OCaml

📢 Note 📢

Multicore OCaml project has now been merged into OCaml 🎉. This repository is no longer developed or maintained. Please follow the updates at the OCaml Github repository.

Citation

If you are citing this work in an academic paper, please cite the ICFP 2020 paper "Retrofitting Parallelism onto OCaml": https://dl.acm.org/doi/10.1145/3408995

---

Branch trunk	Branch 4.14	Branch 4.13	Branch 4.12

README

Overview

OCaml is a functional, statically-typed programming language from the ML family, offering a powerful module system extending that of Standard ML and a feature-rich, class-based object system.

OCaml comprises two compilers. One generates bytecode which is then interpreted by a C program. This compiler runs quickly, generates compact code with moderate memory requirements, and is portable to many 32 or 64 bit platforms. Performance of generated programs is quite good for a bytecoded implementation. This compiler can be used either as a standalone, batch-oriented compiler that produces standalone programs, or as an interactive REPL system.

The other compiler generates high-performance native code for a number of processors. Compilation takes longer and generates bigger code, but the generated programs deliver excellent performance, while retaining the moderate memory requirements of the bytecode compiler. The native-code compiler currently runs on the following platforms:

	Tier 1 (actively maintained)	Tier 2 (maintained when possible)
x86 64 bits	Linux, macOS, Windows, FreeBSD	NetBSD, OpenBSD
x86 32 bits	Linux, Windows	FreeBSD, NetBSD, OpenBSD
ARM 64 bits	Linux, macOS	FreeBSD
ARM 32 bits	Linux	FreeBSD, NetBSD, OpenBSD
Power 64 bits	Linux
Power 32 bits	Linux
RISC-V 64 bits	Linux
IBM Z (s390x)	Linux

Other operating systems for the processors above have not been tested, but the compiler may work under other operating systems with little work.

Copyright

All files marked "Copyright INRIA" in this distribution are Copyright © 1996-2021 Institut National de Recherche en Informatique et en Automatique (INRIA) and distributed under the conditions stated in file LICENSE.

Installation

See the file INSTALL.adoc for installation instructions on machines running Unix, Linux, macOS, WSL and Cygwin. For native Microsoft Windows, see README.win32.adoc.

Documentation

The OCaml manual is distributed in HTML, PDF, and Emacs Info files. It is available at

https://ocaml.org/releases/latest/manual.html

Availability

The complete OCaml distribution can be accessed at

https://ocaml.org/docs/install.html

Keeping in Touch with the Caml Community

There is an active and friendly discussion forum at

https://discuss.ocaml.org/

The OCaml mailing list is the longest-running forum for OCaml users. You can email it at

[email protected]

You can subscribe and access list archives via the Web interface at

https://sympa.inria.fr/sympa/subscribe/caml-list

An alternative archive of the mailing list is also available at

https://inbox.ocaml.org/

There also exist other mailing lists, chat channels, and various other forums around the internet for getting in touch with the OCaml and ML family language community. These can be accessed at

https://ocaml.org/community/

In particular, the IRC channel #ocaml on Libera has a long history and welcomes questions.

Bug Reports and User Feedback

Please report bugs using the issue tracker at https://github.com/ocaml/ocaml/issues

To be effective, bug reports should include a complete program (preferably small) that exhibits the unexpected behavior, and the configuration you are using (machine type, etc).

For information on contributing to OCaml, see HACKING.adoc and CONTRIBUTING.md.

Separately maintained components

Some libraries and tools which used to be part of the OCaml distribution are now maintained separately. Please use the issue trackers at their respective new homes:

• The Graphics library (removed in OCaml 4.09)

• The Num library (removed in OCaml 4.06)

• The OCamlbuild tool (removed in OCaml 4.03)

• The camlp4 tool (removed in OCaml 4.02)

• The LablTk library (removed in OCaml 4.02)

• The CamlDBM library (removed in OCaml 4.00)

• The OCamlWinTop Windows toplevel (removed in OCaml 4.00)

ocaml-multicore's Issues

Installation on OSX

Fails with

ld: illegal thread local variable reference to regular symbol _caml_domain_state

See #56.

Provide a way to indicate that a C function will not access the OCaml heap at all

If [OCaml PR #652][1] or a variant thereof gets merged, there will likely be a significant number of C functions that take only C data types (pointers, ints, unboxed doubles) as arguments, and that never

• Access the OCaml heap
• Call into OCaml code

It should be possible for the compiler to emit code that allows the GC to run during such calls.

Unexpected behaviour

I expected the following program to loop:

effect Foo : (unit -> 'a)

let _ =
  match perform Foo () with
  | x -> ()
  | effect Foo k ->
     continue k (fun () -> perform Foo ())

Instead the effect escapes (exception Unhandled)

If I rewrite it as

effect Foo : (unit -> 'a)

let f () = perform Foo ()

let _ =
  match f () with
  | x -> ()
  | effect Foo k ->
     continue k (fun () -> perform Foo ())

It runs and never terminates.

mark stack overflow

Run queue_test example from reagents library as queue_test.byte 4 1000000. This produces the following output:

[0] items_per_domain = 250000
consumed=189249
consumed=285174
consumed=197374
consumed=328203
mark stack overflow

and terminates.

Provide a way to allocate directly in the major heap

There should be a function (of type () -> 'a) or other way to allocate directly in the major heap. This is useful for data that will definitely be shared with other threads, as well as for mutable data that is read a lot (such as large arrays). Especially in the latter case, it is quite common that performance of reads and writes are critical, while performance of allocation generally isn't.

[Native, Move-based] Implement read barrier in asm code

Read barrier is a GC safe-point under move-based promotion and is also expected to perform minor GCs in the read barrier. Currently, the read barrier is implemented as a C function, whose execution necessitates switching to the system stack and back to OCaml stack for every mutable read. This situation can be improved by implementing the fast path of the read barrier -- result of the read is not in a foreign minor heap -- directly in asm code.

Promote to OCaml trunk

Multicore OCaml is currently based off of vanilla OCaml 4.02.1. The compiler needs to be merged with OCaml 4.03.

Unix.fork

Forking is a disaster in any threaded program, but we'll need to support at least simple uses of Unix.fork.

At first, we should just throw an exception if more than one domain is running when Unix.fork is called. After the fork, the child will need to reinitialise mutexes (e.g. using pthread_atfork), in order to continue running caml code.

Refactor `Field` for move-based promotion

Move-based promotion schemes require that access to accessing fields of mutable objects be safe-points. The idea is to remove Field macro from the C interface and instead provide Mutable_field and Immutable_field macros for accessing mutable and immutable object fields. Mutable_field must be a safe-point.

Installation on Mac OS X Sierra

What I did:

$ git clone https://github.com/ocamllabs/ocaml-multicore
$ cd ocaml-multicore
$ ./configure
$ make world

What went wrong:

../../boot/ocamlrun ../../ocamlc -nostdlib -I ../../stdlib -o extract_crc dynlink.cma extract_crc.cmo
gcc -O  -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -I../../byterun -O  -I../unix -DIN_OCAML_BIGARRAY -DCAML_NAME_SPACE -c bigarray_stubs.c
bigarray_stubs.c:78:3: warning: incompatible pointer types initializing 'value (*)(void)' (aka 'long (*)(void)') with an expression of type 'uintnat (void *)'
      (aka 'unsigned long (void *)') [-Wincompatible-pointer-types]
  caml_ba_deserialize,
  ^~~~~~~~~~~~~~~~~~~
1 warning generated.
gcc -O  -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -I../../byterun -O  -I../unix -DIN_OCAML_BIGARRAY -DCAML_NAME_SPACE -c mmap_unix.c
../../boot/ocamlrun ../../tools/ocamlmklib -oc bigarray bigarray_stubs.o mmap_unix.o ../../byterun/libcamlrun.a 
../../boot/ocamlrun ../../tools/ocamlmklib -o threads st_stubs_b.o -lpthread
ld: illegal thread local variable reference to regular symbol _caml_backtrace_buffer for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [libthreads.a] Error 2
make[2]: *** [otherlibraries] Error 2
make[1]: *** [all] Error 2
make: *** [world] Error 2

I'm not really familiar with ocaml mc, is there a way around this?

Native-backend for Multicore

Multicore OCaml is currently bytecode only. Native-code backend exists for OCaml-effects: ocamllabs/ocaml-effects#1. The native-code backend for Multicore should be roughly similar.

Segmentation fault when assigning function to reference in different thread

The following program

effect Yield : unit

let () =
  let kk = ref (fun () -> failwith "continuation") in
  begin match
    print_endline "0";
    perform Yield;
    print_endline "1";
    Unix.sleep 1;
    perform Yield;
    print_endline "2"
  with
    | () -> print_endline "done"
    | effect Yield k ->
        print_endline "before kk assig";
        kk := (fun () -> continue k ());
        print_endline "after kk assig"
  end;
  print_endline "before spawn";
  Domain.spawn (fun () -> !kk ());
  print_endline "after spawn";
  Unix.sleep 20;
  print_endline "after sleep"

always ends with segmentation fault. The output is usually

0
before kk assig
after kk assig
before spawn
1
after spawn
before kk assig
Segmentation fault

It seems that the problem occurs when performing Yield after Unix.sleep 1. What helps is replacing Domain.spawn (fun () -> !kk ()) by !kk ().

Fibers GCed while still alive

This benchmark [0] in the Reagents library gets stuck as fibers seem to be GCed while still alive. The temporary fix was to incur a space leak and keep alive every fiber [1]. This seems to indicate strongly in the direction that fibers are getting GCed too early.

[0] https://github.com/ocamllabs/reagents/blob/master/test/stack_test_compose.ml
[1] https://github.com/ocamllabs/reagents/compare/avoid-thread-loss

Create a PR for this branch on ocaml/ocaml

KC wrote a blog post here about how to create an opam switch from a local compiler installation, but truthfully it would be much easier to just submit a PR to the ocaml github mirror. OPAM would then have an easy switch for the branch, just as pr132 tracks the Flambda branch.

I didn't want to go ahead and do it myself, though it would only take a minute -- would one of the contributors to this branch want to do it instead?

Error while compiling

Hi,
I get the following error when trying make world:

platform.c: At top level:
platform.c:50:9: error: conflicting types for ‘caml_mem_round_up_pages’
 asize_t caml_mem_round_up_pages(asize_t size)
         ^
In file included from platform.c:3:0:
platform.h:185:9: note: previous declaration of ‘caml_mem_round_up_pages’ was here
 uintnat caml_mem_round_up_pages(uintnat size);
         ^
<builtin>: recipe for target 'platform.o' failed
make[2]: *** [platform.o] Error 1
make[2]: Leaving directory '/home/ocaml-multicore/byterun'
Makefile:187: recipe for target 'coldstart' failed
make[1]: *** [coldstart] Error 2
make[1]: Leaving directory '/home/ocaml-multicore'
Makefile:135: recipe for target 'world' failed
make: *** [world] Error 2

The following is the output given by the configure:

Configuration for the bytecode compiler:
        C compiler used........... gcc
        options for compiling..... -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT
        options for linking.......  -Wl,-E  -lm  -ldl -lcurses -lpthread
        shared libraries are supported
        options for compiling..... -fPIC -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT
        command for building...... gcc -shared -o lib.so -Wl,-rpath,/a/path objs
Configuration for the native-code compiler:
        hardware architecture..... i386
        OS variant................ linux_elf
        C compiler used........... gcc
        options for compiling..... -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT
        options for linking.......   -lm
        assembler ................ as
        preprocessed assembler ... gcc -c
        assembler supports CFI ... yes
        with frame pointers....... no
        naked pointers forbidden.. no
        native dynlink ........... true
        profiling with gprof ..... supported
Source-level replay debugger: supported
Additional libraries supported:
        unix str num dynlink bigarray threads graph
Configuration for the "num" library:
        target architecture ...... ia32 (asm level 2)
Configuration for the "graph" library:
        options for compiling .... 
        options for linking ...... -lX11 

PS: Is there a particular mailing list for this project rather than the general caml-list?
Thanks.

Support for ocamlopt?

Is the current version supposed to have a native compiler? I don't get one when I try to build using opam switch , and then reagents seems to force a native build in the makefile, which fails.

Benchmark various promotion schemes

Benchmark various promotion schemes

1. copy-based promotion: Promote by making a copy of the promoted object in the major heap. Requires write faults to mirror writes to both copies: https://github.com/ocamllabs/ocaml-multicore/tree/master
2. move-forward promotion: Promote by moving the object, and forward any local references to the promoted object. Mutable reads must be safe-points: https://github.com/ocamllabs/ocaml-multicore/tree/move-based-promotion
3. move-gc promotion: Promote by performing a minor GC. Mutable reads must be safe-points: https://github.com/ocamllabs/ocaml-multicore/tree/promote-with-minor-gc
4. mutable-in-major promotion: Allocate all mutable objects and stacks in the major heap. Promote by copy, but does not require write faults since there are no mutable objects in the minor heap.
5. synchronized minor GCs: Heap invariant that prohibits minor to minor pointers is not necessary since the minor GCs are synchronized. This is essentially a stop-the-world parallel minor GC.
6. synchronized minor GCs on read-fault cycles: A variant of move-based promotion that synchronizes the minor GCs of domains that become part of read-fault cycles. The main advantage is that mutable reads need not be safe-points. Very tricky. synchronized minor GCs on read-faults: A variant of 5. Minor GCs become coupled on read-faults. There is no promotion (and the associated problems). Minor GCs are independent if there are no read-faults.

Gc.print_stat segfaults

# Gc.print_stat stdout;;
Segmentation fault: 11

It is quite possible that print_stat hasn't been updated to work with multicore.

Segfault on topkg install

opam install topkg segfaults. The version number is 0.8.1. Here is the error log from ocamlrund:

kc@ubuntu:~/.opam/4.02.2+local-git-master/build/topkg.0.8.1$ ocamlrund ocaml pkg/pkg.ml build --pkg-name topkg --pinned false
[-1] Opening bytecode executable /home/kc/.opam/4.02.2+local-git-master/bin/ocamlrund
[-1] Not a bytecode executable
[-1] Opening bytecode executable /home/kc/.opam/4.02.2+local-git-master/bin/ocamlrund
[-1] Not a bytecode executable
[-1] Opening bytecode executable /home/kc/.opam/4.02.2+local-git-master/bin/ocaml
[-1] Initial stack limit: 8192k bytes
[00] caml_promote: forcing minor GC. %_minor_to_scan=24.574661
[00] Minor collection of domain 0 starting
[00] Mark stack overflow. Postponing 2 pools (24.6%, leaving 12349).
[00] Mark stack overflow. Postponing 3 pools (26.9%, leaving 11978).
[00] Minor collection of domain 0 completed: 99% of 505 KB live, 26085 pointers rewritten
[00] Minor collection of domain 0 starting
[00] Mark stack overflow. Postponing 3 pools (25.0%, leaving 12292).
[00] Mark stack overflow. Postponing 2 pools (20.6%, leaving 13013).
[00] Minor collection of domain 0 completed: 36% of 2048 KB live, 10463 pointers rewritten
[00] Pooled memory: 286720 alloced, 97902 free, 1204 fragmentation
[00] Large memory: 56479 alloced, 0 free, 62 fragmentation
[00] Redarkening pool 0x7f26dc3a8000 (9 others left)
[00] Redarkening pool 0x7f26dc448000 (8 others left)
[00] Redarkening pool 0x7f26dd528000 (7 others left)
[00] Redarkening pool 0x7f26dc438000 (6 others left)
[00] Redarkening pool 0x7f26dc430000 (5 others left)
[00] Redarkening pool 0x7f26dc440000 (4 others left)
[00] Redarkening pool 0x7f26dc458000 (3 others left)
[00] Redarkening pool 0x7f26dc470000 (2 others left)
[00] Redarkening pool 0x7f26dc468000 (1 others left)
[00] Redarkening pool 0x7f26dd530000 (0 others left)
[00] Major slice: 243751 alloc, 1279692 work, 0 sweep, 214036 mark (53085 blocks)
[00] Contributing 256 heaps to GC
[00] Pooled memory: 286720 alloced, 97902 free, 1204 fragmentation
[00] Large memory: 56479 alloced, 0 free, 62 fragmentation
[00] Minor collection of domain 0: skipping
[00] Finished marking major heap. Marked 53085 blocks
[00] GC cycle completed (heap cycled)
[00] Cycling heap [00]
[00] Minor collection of domain 0: skipping
[00] Growing global data to 5632 entries
Segmentation fault (core dumped)

Marshalling and unmarshalling

Marshalling has various problems in multicore (the current implementation has just enough hacks to get the compiler to bootstrap). Specifically:

• Marshalling of closures is broken
  Code pointers are treated specially by marshalling. In stock OCaml, code pointers are detected by using the page table. In multicore, there is no page table. Some changes to the closure representation are needed to allow the marshalling code to reliably detect code pointers.

• Marshalling of Custom_tag objects is strange Fixed by #224 and ocaml/ocaml#1683
  The API for marshalling Custom_tag objects is a bit odd: the format does not store the length and custom unmarshallers are passed a large memory region of unknown size in which to unmarshal. This works in stock OCaml because the unmarshaller knows about the heap format, but it doesn't work in multicore. Multicore requires either API changes (done, but not all clients updated), or a marshal format that stores length (probably a better option, since the old API can be kept)

• Marshalling of concurrently-modified objects is unsafe
  There's no particular care taken at the moment to correctly marshal objects being mutated on a different domain. I don't expect the marshaller to give sensible output in this case, but it shouldn't segfault (the current marshaller can crash, in principle, if other domains modify objects being marshalled and a full GC cycle occurs during marshalling, but this is hard to trigger).

Some of the changes to marshalling might be worth upstreaming to stock OCaml independently of multicore.

Benchmark suite for multicore programs

Given that we are building a multicore compiler, we need to (slowly) build a suite of multicore programs. Primarily to benchmark the programs.

ocamldebug

The bytecode ocaml debugger does not know about effects or multiple stacks.

Implement default handlers

Default handlers have been implemented for bytecode in ocaml-effects: https://github.com/lpw25/ocaml-effects/tree/default-handlers

Optimize noalloc C calls

A noalloc extcall can be made more efficient by instead of saving and restoring %r10 to the OCaml stack, using a register preserved by C calls (e.g. %r13) and marking it destroyed_at_oper in proc.ml.

`make opt` fails after ce29581f2c

$ make world
...
$ make opt
...
../boot/ocamlrun ../ocamlopt -nostdlib -I ../stdlib -for-pack Ocamlbuild_pack -warn-error A -w L -w R -w Z -I ../otherlibs/unix -safe-string -c ocamlbuild.ml
../boot/ocamlrun ../ocamlopt -nostdlib -I ../stdlib -I ../otherlibs/unix -o ocamlbuild.native \
          unix.cmxa ocamlbuild_pack.cmx ocamlbuild_plugin.cmx ocamlbuild_executor.cmx ocamlbuild_unix_plugin.cmx ocamlbuild.cmx
../stdlib/libasmrun.a(minor_gc.o): In function `promote_stack':
/home/samir/src/ocaml-multicore/asmrun/minor_gc.c:109: undefined reference to `caml_clean_stack_domain'
collect2: error: ld returned 1 exit status
...

Quiescent phase in major GC

Avoid starting a new major GC immediately after the previous one ends. Implement quiescent phase for major GC. See 23e5bf0#commitcomment-12341271.

Should `verify_swept` be DEBUG only?

caml_sweep invokes verify_swept [1]. But the name suggests that it should only be invoked when debugging.

[1] https://github.com/ocamllabs/ocaml-multicore/blob/master/byterun/shared_heap.c#L352

Preemptive multithreading

Currently, we can only describe cooperative schedulers, where the fiber voluntarily yields control. In many cases, preemptive multi-threading makes sense. The standard way to do preemptive multi-threading in a system with user-level schedulers is to install an interval timer, handle the timer signal where the scheduler yields control to another runnable fiber. The equivalent code in OCaml might look something like:

open Sys
set_signal sigalrm (Signal_handle (fun _ ->
  let k = (* Get current continuation *) in 
  Sched.enqueue k;
  let k' = Sched.dequeue () in
  (* Set current continuation to k' *)));;

Unix.setitimer interval Unix.ITIMER_REAL

The trouble is that this solution is not compositional i.e) it breaks the nice abstraction that algebric effect handlers provide. For one, it is unclear where the signal handler actually runs. We somehow need to get hold of the current continuation (fiber) and switch control to a different fiber. However, the only way to get hold of a continuation in our system is at the handler.

Ultimately, the signal handler is a callback. Algebric effect handlers help us avoid callback-oriented programming and instead express the same program in direct-style. Consider a TimerInterrupt effect:

effect TimerInterrupt : unit
  with function TimerInterrupt -> ()

whose default behaviour is to do nothing. We can now implement preemptive multithreading simply as a case in the scheduler handler:

let rec spawn f = 
  match f () with
  | () -> dequeue ()
  | effect Yield k -> yield k
  ...
  | effect TimerInterrupt k -> yield k

and yield k = enqueue k; dequeue ()

Incorrect initialisation of large immutable objects

If an immutable object is bigger than Max_young_wosize, its initialisation will create invalid major -> minor immutable fields.

Switch to Yuasa barrier from Dijkstra barrier

This eliminates the small stop-the-world phase at the end of a major collection to finish the marking work. Stop-the-world phase is only required to flip the colors at the end of collection.

Finalizers and resource cleanup

• What is the behavior of finalizers attached to continuations?
• How are fibers that become unreachable cleanup its resources?

This is equivalent to ocamllabs/ocaml-effects#13.

Abstract_tag blocks are incompatible with copy-based promotion

The current strategy for sharing local heap objects is by making a copy of the object in the major heap. The runtime ensures that the writes performed to either copies of the promoted object is mirrored to the copy using a write fault mechanism. However, this design is incompatible with Abstract_tag blocks.
C bindings use blocks with tag Abstract_tag as raw memory chunks, and read and write from them as raw C structures (which they often are). It'll be pretty hard to intercept such writes and mirror them in a copy or fault on them.

A workaround to this problem is to enforce the constraint that Abstract_tag blocks are only allocated in the major heap. In this case, caml_alloc_small(size, Abstract_tag) would be a runtime error. Instead, a new function caml_alloc_abstract(size) may be provided, which allocates directly in the major heap.

Question about schedulers

OK, so here's a concern I've had since the day the in-language schedulers were announced: the scheduler is a critical piece of code, right? You really want it to be as efficient as possible, and OCaml has non-trivial overheads. So I envision a company like Jane Street realizing that to maximize their performance, they need to use a custom C-based scheduler. They'd be essentially enhancing the runtime with their own scheduler, and to get maximum performance from OCaml, you'd have to compile against this additional bit of C code. So even though we're working to move the scheduler into the user code, it'll de-facto be a part of the non-standard runtime. Is this something we want to avoid? Would it make sense to bundle standard schedulers with the compiler -- is this something that's planned?

No stack trace when compiling with option `-g'

The following program false.ml:

let _  = assert false

does not admit a stack trace when compiled using the debug option -g, e.g.

$ ocamlopt -v
The OCaml native-code compiler, version 4.02.2+multicore-dev0
Standard library directory: /home/dhil/.opam/4.02.2+local-git-fix-sys-threads/lib/ocaml
$ ocamlopt -g false.ml
$ ./a.out
Fatal error: exception Assert_failure("false.ml", 1, 8)

Although, a stack trace is produced when compiled with the 4.02.3 OCaml compiler:

$ ocamlopt -v 
The OCaml native-code compiler, version 4.02.3
Standard library directory: /home/dhil/.opam/4.02.3/lib/ocaml
$ ocamlopt -g false.ml
$ ./a.out
Fatal error: exception Assert_failure("false.ml", 1, 8)
Raised at file "false.ml", line 1, characters 8-20

failwith silently fails on non-main domains

let r = ref false

let () = Domain.spawn ( fun () ->
  Printf.printf "Before failwith\n%!";
  failwith "Hello";
  r := true)

let rec loop () = if !r then () else loop ()

prints Before failwith and the program runs to completion. Must print the error message Fatal error: exception Failure("Hello").

Consider non-moving generational mark-region for the shared heap

One option for the shared heap is non-moving generational mark-reagion GC. The advantage of this is that it is faster than many other non-moving algorithms (it is the basis for the "New Garbage Collector" for LuaJIT) and might be fast enough to allow directly allocating some mutable objects on the major heap.

[Copy-based, Native] Deadlock in chameneos-redux

chameneos-redux benchmark intermittently deadlocks under native backend of copy-based promotion scheme. Here are the steps to recreate the bug:

# Assuming `native` branch is the current compiler
$ git clone https://github.com/kayceesrk/ocaml15-eff
$ cd ocaml15-eff
$ git checkout multicore
$ cd chameneos-redux
$ make chameneos-eff
$ ./chameneos-eff 1000 2

After a few runs, you may see that the program deadlocks. On investigation, it appears that one of the worker threads spawned goes AWOL during the execution, which blocks the main thread from running to completion.

Effects in local modules behave differently if they come from a functor

Heya,

It seems that "effect instances" (or whatever the thing is called that is compared for equality when you perform) work differently depending on whether they're defined in a module or in a functor: for modules they are global names; for functors they seem to be generated dynamically when you instantiate the functor. This was surprising to me. Is this the expected behaviour?

To reproduce, run the below code. Even though the module and the functor are implemented the same way, the output is different:

Parametrised: A remembers: a
Parametrised: B remembers: b
Parametrised: C remembers: c
Unparametrised: A remembers: d
Unparametrised: B remembers: d
Unparametrised: C remembers: d

Reproduce using this source:

open Printf

module type Wrapping = sig

  val remembering : string -> (unit -> unit) -> unit
  val say : unit -> unit

end

module type UNIT = sig end

(* Two pretty identical modules *)

module Parametrised (U : UNIT) : Wrapping = struct

  effect Effect : unit -> unit;;

  let remembering s f = match f () with
    | x -> x
    | effect (Effect ()) k -> printf "%s\n" s; continue k ()

  let say () = perform (Effect ())

end

module Unparametrised : Wrapping = struct

  effect Effect : unit -> unit;;

  let remembering s f = match f () with
    | x -> x
    | effect (Effect ()) k -> printf "%s\n" s; continue k ()

  let say () = perform (Effect ())

end

let _ =
  let module A = Parametrised (struct end) in
  let module B = Parametrised (struct end) in
  let module C = Parametrised (struct end) in
  let module D = Parametrised (struct end) in
  A.remembering "a" (fun _ ->
    B.remembering "b" (fun _ ->
      C.remembering "c" (fun _ ->
        D.remembering "d" (fun _ ->
          printf "Parametrised: A remembers: "; A.say ();
          printf "Parametrised: B remembers: "; B.say ();
          printf "Parametrised: C remembers: "; C.say ();
          ()))))


let _ =
  let module A = Unparametrised in
  let module B = Unparametrised in
  let module C = Unparametrised in
  let module D = Unparametrised in
  A.remembering "a" (fun _ ->
    B.remembering "b" (fun _ ->
      C.remembering "c" (fun _ ->
        D.remembering "d" (fun _ ->
          printf "Unparametrised: A remembers: "; A.say ();
          printf "Unparametrised: B remembers: "; B.say ();
          printf "Unparametrised: C remembers: "; C.say ();
          ()))))

Is this a feature or a bug?

Refactor `DELEGATE` to `REPERFORM`

and refactor delegate to reperform, which is more consistent with reraise.

[yuasa barrier] Avoid mutator blocking on stack scanning

Under yuasa (deletion) barrier design, the all the fiber stacks need to be marked at the start of the GC cycle. We can perform this incrementally by scanning the non-black fiber stack on context switching to it. However, this scanning operation still blocks the mutator and can add noticeable latency for large stacks.

This proposal [0] for Golang obviates STW stack rescanning by enhancing the write barrier to take into account whether the current stack is black. This design avoids the non-incremental stack scanning in the current design by paying instead for slower write barrier.

The main advantage with the scan-on-context-switch design is that non-concurrent OCaml code does not pay any additional cost (no need to check for colour of the current stack in the write barrier). I would think this to be important for performance backwards compatibility. The disadvantage is that context switching may incur a stack scan. This seems like a reasonable choice since we expect fewer context switches than writes. The stack scanning on context switching is only needed once per major GC cycle, that too if the stack hasn’t already been scanned by the GC.

Since the change to get new write barrier design is minimal, it would be worthwhile to implement and benchmark this scheme.

[0] https://github.com/golang/proposal/blob/master/design/17503-eliminate-rescan.md

`Unhandled` exception should print the effect name

# effect E : int -> unit;;
effect E : int -> unit
# perform (E 0);;
Exception: Unhandled.

Currently, unhandled effects raise Unhandled exception at the top level. It would be useful to change exception Unhandled to exception Unhandled of string where the exception is parameterized with the unhandled effect name.

`make -j world` fails intermittently

make -j world works fine on the vanilla OCaml.

Add `Gc.promote` to explicitly promote objects to the major heap.

Gc.promote : 'a -> unit exposes the ability to explicitly perform promotions. In move-based promotion, promoting recently allocated objects is cheaper than promoting older objects since a smaller part of the young heap needs to be scanned in the former case. The behavior of Gc.promote x is the following:

• if x is a primitive or a major heap object, then Gc.promote x is a noop.
• if x is a young object, then x (and its transitive closure) is promoted to the major heap.

Since promotion needs to forward all reference to promoted objects, Gc.promote must be a safe-point where all the roots are visible to the GC. This must be explicitly ensured by the C bindings.

"Continuation already used"

# effect E1 : unit
  effect E2 : unit

  let handle f =
    try f () with effect E1 k -> continue k ()

  let () = handle
      (fun () ->
         try for i = 0 to 196605 do perform E1 done
         with effect E2 k -> ())
   ;;
Exception: Invalid_argument "continuation already used".

Lots of warnings generated for `static_assertion_failure_line` on OSX

The same set of warnings are generated for every file. gcc is:

$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.4.0
Thread model: posix

Maybe it is just an issue with clang. Haven't tried with gcc.

gcc -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -I../../byterun -O   -c accept.c
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:1:1: warning: redefinition of typedef 'static_assertion_failure_line_1' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(0, volatile uintnat, young_limit)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_1
^
../../byterun/domain_state.tbl:1:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:142:1: note: expanded from here
static_assertion_failure_line_1
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:2:1: warning: redefinition of typedef 'static_assertion_failure_line_2' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(1, char*, young_ptr)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_2
^
../../byterun/domain_state.tbl:2:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_2
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:3:1: warning: redefinition of typedef 'static_assertion_failure_line_3' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(2, char*, young_start)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_3
^
../../byterun/domain_state.tbl:3:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_3
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:4:1: warning: redefinition of typedef 'static_assertion_failure_line_4' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(3, char*, young_end)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_4
^
../../byterun/domain_state.tbl:4:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_4
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:5:1: warning: redefinition of typedef 'static_assertion_failure_line_5' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(4, char*, top_of_stack)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_5
^
../../byterun/domain_state.tbl:5:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_5
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:6:1: warning: redefinition of typedef 'static_assertion_failure_line_6' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(5, char*, bottom_of_stack)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_6
^
../../byterun/domain_state.tbl:6:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_6
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:7:1: warning: redefinition of typedef 'static_assertion_failure_line_7' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(6, uintnat, last_return_address)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_7
^
../../byterun/domain_state.tbl:7:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_7
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:8:1: warning: redefinition of typedef 'static_assertion_failure_line_8' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(7, value*, gc_regs)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_8
^
../../byterun/domain_state.tbl:8:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_8
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:9:1: warning: redefinition of typedef 'static_assertion_failure_line_9' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(8, char*, exception_pointer)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_9
^
../../byterun/domain_state.tbl:9:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_9
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:10:1: warning: redefinition of typedef 'static_assertion_failure_line_10' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(9, intnat, backtrace_pos)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_10
^
../../byterun/domain_state.tbl:10:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_10
^
In file included from accept.c:17:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:11:1: warning: redefinition of typedef 'static_assertion_failure_line_11' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(10, intnat, backtrace_active)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_11
^
../../byterun/domain_state.tbl:11:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_11
^
11 warnings generated.
gcc -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -I../../byterun -O   -c access.c
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:1:1: warning: redefinition of typedef 'static_assertion_failure_line_1' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(0, volatile uintnat, young_limit)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_1
^
../../byterun/domain_state.tbl:1:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:142:1: note: expanded from here
static_assertion_failure_line_1
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:2:1: warning: redefinition of typedef 'static_assertion_failure_line_2' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(1, char*, young_ptr)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_2
^
../../byterun/domain_state.tbl:2:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_2
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:3:1: warning: redefinition of typedef 'static_assertion_failure_line_3' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(2, char*, young_start)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_3
^
../../byterun/domain_state.tbl:3:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_3
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:4:1: warning: redefinition of typedef 'static_assertion_failure_line_4' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(3, char*, young_end)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_4
^
../../byterun/domain_state.tbl:4:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_4
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:5:1: warning: redefinition of typedef 'static_assertion_failure_line_5' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(4, char*, top_of_stack)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_5
^
../../byterun/domain_state.tbl:5:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_5
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:6:1: warning: redefinition of typedef 'static_assertion_failure_line_6' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(5, char*, bottom_of_stack)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_6
^
../../byterun/domain_state.tbl:6:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_6
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:7:1: warning: redefinition of typedef 'static_assertion_failure_line_7' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(6, uintnat, last_return_address)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_7
^
../../byterun/domain_state.tbl:7:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_7
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:8:1: warning: redefinition of typedef 'static_assertion_failure_line_8' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(7, value*, gc_regs)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_8
^
../../byterun/domain_state.tbl:8:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_8
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:9:1: warning: redefinition of typedef 'static_assertion_failure_line_9' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(8, char*, exception_pointer)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_9
^
../../byterun/domain_state.tbl:9:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_9
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:10:1: warning: redefinition of typedef 'static_assertion_failure_line_10' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(9, intnat, backtrace_pos)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_10
^
../../byterun/domain_state.tbl:10:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_10
^
In file included from access.c:16:
In file included from ../../byterun/memory.h:27:
In file included from ../../byterun/domain.h:5:
In file included from ../../byterun/domain_state.h:25:
../../byterun/domain_state.tbl:11:1: warning: redefinition of typedef 'static_assertion_failure_line_11' is a C11 feature [-Wtypedef-redefinition]
DOMAIN_STATE(10, intnat, backtrace_active)
^
../../byterun/domain_state.h:24:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(offsetof(struct caml_domain_state, name) == idx * sizeof(void*));
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_11
^
../../byterun/domain_state.tbl:11:1: note: previous definition is here
../../byterun/domain_state.h:23:3: note: expanded from macro 'DOMAIN_STATE'
  CAML_STATIC_ASSERT(sizeof(caml_domain_state->name) == sizeof(void*)); \
  ^
../../byterun/domain_state.h:19:31: note: expanded from macro 'CAML_STATIC_ASSERT'
#define CAML_STATIC_ASSERT(b) CAML_STATIC_ASSERT_2(b, __LINE__)
                              ^
../../byterun/domain_state.h:18:36: note: expanded from macro 'CAML_STATIC_ASSERT_2'
#define CAML_STATIC_ASSERT_2(b, l) CAML_STATIC_ASSERT_3(b, l)
                                  ^
../../byterun/domain_state.h:17:16: note: expanded from macro 'CAML_STATIC_ASSERT_3'
  typedef char static_assertion_failure_line_##l[(b) ? 1 : -1]
              ^
<scratch space>:144:1: note: expanded from here
static_assertion_failure_line_11
^
11 warnings generated.  

Unhandled raised when two domains use single function

The following code works as expected

effect Print : string -> unit

let run thunk =
  match thunk () with
  | () -> ()
  | effect (Print s) k -> print_endline s; flush stdout; continue k ()

let thunk () =
  let rec write x =
    Printf.printf "domain %d %s\n" (Domain.self ()) x;
    flush stdout;
    perform (Print x);
    Unix.sleep 1;
    write x in

  (* Same as [write]. *)
  let rec write2 x =
    Printf.printf "domain %d %s\n" (Domain.self ()) x;
    flush stdout;
    perform (Print x);
    Unix.sleep 1;
    write2 x in

  Domain.spawn (fun () -> run (fun () -> write "first"));
  write2 "second"

let () = run thunk

but when I remove write2 and replace write2 "second" by write "second" then the exception Unhandled is raised when performing effect Print "second".

Remove profiling slop space from fibers

Currently, a kilobyte of slop space is used when (a) signal handlers are turned on or (b) profiling is enabled. Signal handlers should use a separate stack, allocated with sigaltstack at startup. It turns out that mcount is a short assembly wrapper that calls the C function mcount_internal passing the current PC and return address (see sysdeps/x86_64/_mcount.S in glibc source), so we can handle mcount as a standard C call (switching to the system stack) if we call mcount_internal instead.

See also, #47.

The latest two commits brokes the build

cbb60bb1689601f6fe0f782f046330b4bea51337 and a36d7ae41ff0099aa409476cfce960a458dff1ae actually brokes the build. It was complaining about lack of minor_heap.h.

Segmentation fault with reagent queues

The example is queue_test.ml under ocaml-reagents [1]. The segfault can be replicated by:

git clone https://github.com/kayceesrk/ocaml-reagent.git
cd ocaml-reagent
git checkout mc-sched-segfault
make
./queue_test.byte 8 100000

Running under debug mode leads to a variety of error conditions. It either (1) gets stuck with the following output:

[01] Handling read fault for domain [01]
[04] Read fault returned (0x1022bd260)
[01] Read fault to domain [04]
[04] Handling read fault for domain [04]
[01] Read fault returned (0x10228cc70)
[05] Major slice: 226046 alloc, 1186741 work, 12289 sweep, 89140 mark (29708 blocks)
[05] Slow spin-wait loop in stw_phase at domain.c:436
[04] Slow spin-wait loop in stw_phase at domain.c:436
[07] Slow spin-wait loop in stw_phase at domain.c:436

or terminates just after starting a major slice collection (2):

[02] Major slice: 262143 alloc, 1376250 work, 0 sweep, 417685 mark (139228 blocks)
[03] Contributing 255 heaps to GC
[02] Contributing 1 heaps to GC
[03] GCing inactive domain [00]
[03] Pooled memory: 331776 alloced, 93674 free, 359 fragmentation
[03] Large memory: 4958 alloced, 0 free, 8 fragmentation
[02] Pooled memory: 1978368 alloced, 12907 free, 483 fragmentation
[02] Large memory: 4099 alloced, 0 free, 2 fragmentation
file major_gc.c; line 114 ### Assertion failed: v == mark_normalise(v)
Abort trap: 6

or terminates with (3):

[04] Major slice: 262143 alloc, 1376250 work, 16385 sweep, 1074197 mark (358060 blocks)
[05] Read fault returned (0x10da8f430)
[05] Contributing 251 heaps to GC
[04] Contributing 1 heaps to GC
[03] Contributing 1 heaps to GC
[01] Contributing 3 heaps to GC
[01] GCing inactive domain [00]
[01] Pooled memory: 155648 alloced, 93995 free, 316 fragmentation
[01] Large memory: 4958 alloced, 0 free, 8 fragmentation
[03] Pooled memory: 163840 alloced, 27669 free, 40 fragmentation
[03] Large memory: 4099 alloced, 0 free, 2 fragmentation
[03] Minor collection starting
[03] Minor collection completed: 0 of 840 kb live, 0 pointers rewritten
[05] Pooled memory: 290816 alloced, 33798 free, 71 fragmentation
[05] Large memory: 4099 alloced, 0 free, 2 fragmentation
[05] Minor collection starting
[05] Minor collection completed: 0 of 1474 kb live, 0 pointers rewritten
[04] Pooled memory: 1392640 alloced, 15675 free, 340 fragmentation
[04] Large memory: 4099 alloced, 0 free, 2 fragmentation
[04] Finished marking major heap. Marked 445442 blocks
[03] Finished marking major heap. Marked 153160 blocks
Segmentation fault: 11

or (4):

[05] Minor collection starting
[06] Finished marking major heap. Marked 2736 blocks
[00] Large memory: 4958 alloced, 0 free, 8 fragmentation
[05] Minor collection completed: 0 of 76 kb live, 0 pointers rewritten
[06] GCing inactive domain [07]
[00] Minor collection starting
[06] Pooled memory: 32768 alloced, 10489 free, 8 fragmentation
[05] Finished marking major heap. Marked 4756 blocks
[06] Large memory: 4099 alloced, 0 free, 2 fragmentation
[01] Finished marking major heap. Marked 27399 blocks
[06] Finished marking major heap. Marked 48626 blocks
[00] Minor collection completed: 1472 of 1922 kb live, 0 pointers rewritten
file shared_heap.c; line 371 ### Assertion failed: h && !Has_status_hd(h, global.GARBAGE)
Abort trap: 6

The debug logs point to error introduced by allocation/manipulation of the major heap that is triggered during major collection.

[1] https://github.com/kayceesrk/ocaml-reagent/blob/mc-sched-segfault/test/queue_test.ml

Start a gitter page for easy discussion and questions

See heading. I think this could be useful. For example, I'd like to resolve the thread-local-storage issue on OSX but it'd go much faster if I could just ask some questions and benefit from your knowledge.