[TSan] Random failure of `tests/parallel/domain_dls.ml`

[dustanddreams]

[TSan] Random failure of `tests/parallel/domain_dls.ml`

[OlivierNicole]

So, in principle, these accesses being concurrent should be ok. The question is why our machinery based volatile * accesses failed to make TSan see it that way. The incriminated read is a use of Tag_val, which loads the least significant byte of a header word. My hypothesis is that because s390x is big-endian, the load is not aligned, and we currently consider unaligned volatile accesses to behave like plain accesses. (Tangent: this code I linked to seems to have an superfluous alignment check now that I’m looking at it.)

My feeling is that the unaligned nature of the load does not change anything to the fact that these accesses are safe In Practice™: the GC marking does not modify the tag. So the question is whether there is way to remove this false report.

[dustanddreams]

Well I'd argue that, on platforms requiring strict alignment of data such as s390x (i.e. when allow_unaligned_access = false in arch.ml), loads smaller than the size of a register are atomic (the memory bus transaction is a machine word read, with the proper byte enables set or unset to only return the subset of the word the ALU is interested in).

[gasche]

If I understand @dustanddreams' point, the recommend fix would be to change the definitions of the tsan_volatile_{read,write} functions in tsan.c on certain architectures.

[jmid]

Unsure if this is related, but 4 days ago I observed a crash in our Domain.DLS trunk tests (on 32-bit though):
ocaml-multicore/multicoretests#446
The striking thing to me was that this was in a sequential test even.
I've not had time to look more into reproducing and reporting it.

[OlivierNicole]

Well I'd argue that, on platforms requiring strict alignment of data such as s390x (i.e. when allow_unaligned_access = false in arch.ml), loads smaller than the size of a register are atomic (the memory bus transaction is a machine word read, with the proper byte enables set or unset to only return the subset of the word the ALU is interested in).

Interesting, I don’t know much about unaligned accesses. How are they different in, for instance, amd64?

[dustanddreams]

Interesting, I don’t know much about unaligned accesses. How are they different in, for instance, amd64?

I think we have a misunderstanding of what "unaligned access" means here.

An unaligned access is a memory access of width w at address a, where a is not a multiple of w. Depending on the hardware platform, such an operation may trap, behave differently (e.g. silently accessing a rounded-down address and thus not returning the expected data), or require multiple memory bus transactions to complete (and as such, hardly atomic).

Now if you consider atomicity, aligned read accesses not larger than the width of the memory bus (usually at least as wide as the machine register, but sometimes wider, e.g. the 88110 was a 32-bit processor with a 64-bit memory bus and able to perform atomic loads of 64-bit values into register pairs) are always atomic (well, except on alpha without the BWX extension, but these are not supported by the native compiler since 4.x).

On the other hands, aligned write accesses are atomic only if they have the same width as the memory bus, or if the hardware provides some atomicity guarantees, since smaller-than-width writes will be turned into a 3-step read, modify the subset bytes, write operation. (which is why the x86 architecture has the "LOCK" prefix to require the memory bus to not be released between these steps, in order to appear to be an atomic operation to other devices).

I will work on a relaxed version of __tsan_volatile_read* to demonstrate this.

[dustanddreams]

Fixed by #13067.