slightknack/rust-abi-wiki

A wiki about Rust's ABI and plans to stabilize it. Open to editing, all PRs will be merged.

rust-abi-wiki's Issues

What about the calling convention?

They are a mess and are highly architecture specific. In addition many parts of the calling convention can only be implemented as part of the actual codegen backend like LLVM. This includes register assignment. The Rust ABI builds on top of the native C calling convention, but only uses part of it's functionality. For example #[repr(C)] struct A(u32, u32) would be passed in a single register for the x86_64 System-V call conv, while Rust passes it in memory as pointer. If there is no #[repr(C)] it uses two registers. #[repr(C)] struct BigType([u8; 1024]) would be stored at a fixed stack offset for the x86_64 System-V call conv, while Rust passes a pointer to wherever it exists. Then there are special rules for mixing integers and floating points in a single struct. And another thing is that Rust currently forces vector types to be passed in memory in certain conditions as a workaround for bugs when mixing functions without simd enabled and with simd enabled.

Could loading a Rust dynamic library requires a (very up-to-date) special linker?

When loading a dynamic library, the dynamic linker does some basics things like defining the address of the relocatable code, … For C and C++ those mechanisms are very basics, and standardized since a long time. This allow basically any library compiled with any compiler to be dynamically linked using any dynamic linker, without any regards to the versions of the various tools used.

If we could require a special linker for dynamically linking rust binaries, this would allow to apply custom logic during loading time. Requiring a (very) up-to-date Rust dynamic linker would also allow to change the ABI specification since the new linker would know what to do with it. However, this means that in order to start any binary using a Rust dynamic library, the user would need to have a compatible (ie. up-to-date) dynamic Rust linker.

I see 3 options:

• restrict ourself to the current capabilities of the current linkers.
• create a new linker, with added capabilities, then fix that new capability set in stone (so it would be possible to dynamically link a Rust library compiled in 2030 with a 2021 Rust dynamic linker).
• create a new linker, with added capabilities, and allow ourself to modify the set of capabilities in the future (for example for each Rust edition) which requires the user to update each time (ie. every 3 years with the current release cycle of Rust editions).

Add more references for ABIs

Maybe it's worth adding these links to the reference page

• Microsoft COM
• Swift ABI Design
• Swift vs Rust for ABI

Source for Fuschia Rust Ban

Recently, the Fuschia OS Team at Google decided to ban Rust's for use in Fuschia microkernel

This is weird because it used in Fuschia

Stable Layout and Implementation

a stable ABI involves a lot more than just fixing #[repr(Rust)] in place and deciding on a calling convention, but also making sure every struct is ABI compatible and that functions and traits don't change.

This work is is the kind of work that I think could be done mostly as a library, with a few compiler changes of course. There's a fundamental trade-off between flexibility and stability here, and Rust chose to lean towards flexibility with regards to ABI.

Functions - inlining affords you no flexibility in changing any externally observable behavior of a function (including bug fixes). Swift has precedent here with @inlineable, and we may have to treat everything as #[inline(never)] unless we have the appropriate #[inline].

Traits can be thought of as basically a bunch of functions, but with the caveat that associated consts can never change (this is also a problem in Java, C, and C#).

In the same vein, constants and statics can never change their externally visible value, but I assume that this won't be a big deal in practice.

Structs are likely going to be the biggest problem in practice, because the prevalence of privacy and static linking has lead to a behavior of libraries changing their type representations to evolve. Fixing the representation of types has been partially addressed by the Safe Transmute RFC, but it doesn't really address our needs. Some sort of pimpl idiom/extern type is going to be neccessary I might imagine.

A final note about std and libraries is that we'll either have to pin their versions or switch to libraries that promise a stable ABI themselves (if any of their types are used in our ABI). Pinning std is easiest but runs the risk of incompatible desugarings (not to mention lack of cargo-std support). Pinning libraries might lead to annoying issues in practice with DLL hell. I would hate to have shim libraries that pimpl everything, but they may be the price for dynamic linking...

Make into a readable git Book.

This would be using mdBook, hosted on GitHub pages or the like.

Note Swift's "ABI stability" is per OS

One apparent aspect of Swift's "ABI stability" that doesn't seem to always be explicitly stated in discussions is that "ABI stability is implemented for each operating system that it compiles and runs on"[0] and (as of 2019, at least, I can't find more recent references) the ABI is only "stable for Swift 5 on Apple platforms"[0].

Given that Apple controls the platforms in question, that's a fairly significant difference to Rust's situation and seems worth noting in related discussion.

Some resources with more details:

• "How Swift Achieved Dynamic Linking Where Rust Couldn't": https://gankra.github.io/blah/swift-abi/

• "ABI Stability and More": https://swift.org/blog/abi-stability-and-more/

• "Evolving Swift On Apple Platforms After ABI Stability": https://swift.org/blog/abi-stability-and-apple/

---

[0]:

It has been a longstanding goal to stabilize Swift’s ABI on macOS, iOS, watchOS, and tvOS. [...]

You may ask: what about other platforms? ABI stability is implemented for each operating system that it compiles and runs on. Swift’s ABI is currently declared stable for Swift 5 on Apple platforms. As development of Swift on Linux, Windows, and other platforms matures, the Swift Core Team will evaluate stabilizing the ABI on those platforms.

-- Source: https://swift.org/blog/abi-stability-and-more/ (Emphasis added.)

Enforcing ABI only at the boundary (both entry and exit points)

I had a proposal in the discussion on IRLO that (I think) was interesting. It started with this comment. I didn't see it in your sum-up.

Here is the gist of my idea: the only place where this ABI are the entry points of your code (main for a binary, any publicly exported function for a library), and the exit point (when calling external library). Those boundary must use the same ABI that whatever is used on the other side. This include struct layout, calling convention, … However, anything in between can use any add-hoc ABI.

Example: we have the following call-graph (a -> b means that a is calling b):

f1 -> f2 -> f3 -> g1 -> g2 -> g3 -> h1 -> h2

Where f1, g1 and h1 are pub function respectively exported by the library F, G, and H. The other functions are not publicly visible (they are internal to their respective libraries).

First, we need to compile H. During compilation the ABI of the public functions is going to be fixed. Lets name it ABI_H. Internally the ABI doesn't need to be ABI_H, as long as it's a compatible ABI with a mechanic transformation (for example changing the endianness of a number). Let's name it ABI_h.

Then the library G will be compiled. The calling convention, and the layout of all the objects passed to h1() by g3() must match the ABI_H ABI. Internally the compiler could be allowed to do the appropriate transformation to the types if they don't match , or report a build error if the transformation is not possible. Once again, the ABI of the public funtions of G will have to be fixed. Lets name it ABI_G, and ABI_g for the internal ABI.

Finally F is compiled and linked against G, and the same reasoning is applied. Let's name the internal ABI ABI_f.

To sum-up, the functions must communicate with the following schema:

f1 -> ABI_f -> f2 -> ABI_G -> g1 -> ABI_g -> g2 -> ABI_g -> g3 -> ABI_H -> h1 -> ABI_h -> h2

In order to have all of this working, the only think needed is that ABI_f mast be compatible with ABI_G, itself with ABI_g, itself with ABI_H, and finally itself with ABI_h. Technically a single crate could use multiple internal ABI. And as you can see, there is no assumption on which languages were used or compiled to write F, G and H, the only important thing is how to be able to consume those library. If all ABI are the same (like the C ABI, Swift ABI, or the Rust ABI) this obviously works, and doesn't need any transformation.

To conclude all of this, the only place where ABI need to be enforced is:

• entry points: publicly visible items (function, object, …)
• exit points: when calling external functions (or syscalls, interruptions, …)

This also means that a crate could be compiled multiple times with different public ABI (in order to export itself with the C ABI for a C consumer or the Swift ABI for a Switft consumer) without any change in the source code. If the public ABI is incompatible with the types used (like Result<T> for the C ABI) it would result in a compile time error.

Crate metadata

There are many benefits a standardized ABI would bring to Rust. A stable ABI enables dynamic linking between Rust crates, which would allow for Rust programs to support dynamically loaded plugins (a feature common in C/C++). Dynamic linking would result in shorter compile-times and lower disk-space use for projects, as multiple projects could link to the same dylib. For example, imagine having multiple CLIs all link to the same core library crate.

To be able to link to the same libstd using multiple rustc versions the crate metadata format would need to be stabilized. This in turn would require the stabilization of all nightly features used by libstd, as otherwise a different rustc version doesn't know what to do when it finds a reference to an unstable feature. It can't just skip it, as for example macros defined in libstd are able to use unstable features even when the macro itself is stable.