Removing build-time dependency on bindgen

[tov]

Removing build-time dependency on bindgen

[yorickpeterse]

@MarkMcCaskey The pull request #37 removes the need for bindgen, and contains instructions on how to apply these patches (e.g. for testing). Please give that a try to see if it works for you, thanks!

[MarkMcCaskey]

@yorickpeterse

Thanks! I wasn't using the no-bindgen-patch branch for libffi, just libffi-sys, that's probably why it failed. That works with the system feature on my set up now.

What causes the error I'm not sure, but make sure that you have both automake and autoconf installed; not sure if any specific versions are required either.

It's caused by OSX permissions since version 10.15. For whatever reason it works perfectly fine in a normal directory but attempting to build it on an external drive causes it to fail. I've disabled all the protections I could find and it still happens 🤷 ; it's not related to libffi-rs or libffi at all as far as I can tell, it may be a bug in OSX.

---

The change looks great though -- thanks for making this work!

[tov]

Hey, thanks for the suggestion! I believe we could do it—we’d just have to provide the Bindgen output for each target that we wanted to support in that mode. Libffi depends on some fiddly ABI details, so I don’t know what granularity of target we’d need to distinguish. And I’m not sure whether it makes sense to ship a bunch of versions in one crate, or have per-target crates as optional dependencies.

I’m also not sure about how to set up CD to do the necessary preprocessing and packaging.

Thoughts?

[yorickpeterse]

One possibility is to include pre-generated bindings for some platforms (e.g. Linux, macOS, and Windows), but fall back to using bindgen at install-time for other platforms. If needed, I can provide the output of generated.rs for 64 bits Linux, macOS, and Windows.

[yorickpeterse]

I generated the generated.rs files for three platforms: Linux 64 bits, macOS 10.13 64 bits, and Windows 10 64 bits in an MSYS2/MINGW64 shell. I then formatted the files using rustfmt. The files are as follows:

Platform	Triple	File
MSYS2/MINGW64	x86_64-windows-gnu	x86_64-windows-gnu.rs
macOS 10.13	x86_64-macos-unknown	x86_64-macos-unknown.rs
Linux 5.8	x86_64-linux-gnu	x86_64-linux-gnu.rs

Some notes:

1. There are differences between these files, such as the defined constants.
2. On Windows all code was crammed on only two lines for some reason.
3. I don't have a Visual Studio setup, so I can't provide a generated.rs file for that at this time
4. For MSYS2 I used the clang executable from the mingw-w64-x86_64-clang package

[yorickpeterse]

Looking at the output of the files above (after removing tests), it seems the libffi API is the same across these platforms. The differences in turn seem to be caused by bindgen noise: types/aliases that it parses but aren't actually used anywhere.

If the public libffi API is indeed the same across platforms, we could get rid of bindgen/libclang/etc entirely. I'll see how far I can get this going.

[MarkMcCaskey]

I tried it locally and I'm running into a bit of a strange issue:

error: failed to run custom build command for `libffi-sys v0.9.2-alpha.0 (/Users/mark/libffi-sys-rs)`

Caused by:
  process didn't exit successfully: `/Volumes/external/wasmer/target/release/build/libffi-sys-df1fe7bf7d4b05f3/build-script-build` (exit code: 101)
  --- stderr
  autoreconf: Entering directory `.'
  autoreconf: configure.ac: not using Gettext
  autoreconf: running: aclocal -I m4
  autoreconf: configure.ac: tracing
  autoreconf: configure.ac: not using Libtool
  autoreconf: running: /usr/local/Cellar/autoconf/2.69/bin/autoconf
  autoreconf: configure.ac: not using Autoheader
  autoreconf: configure.ac: not using Automake
  autoreconf: Leaving directory `.'
  configure: line 1: syntax error near unexpected token `('
  configure: line 1: `m4trace:aclocal.m4:15: -1- m4_include([acinclude.m4])'
  thread 'main' panicked at 'Configuring libffi', /Users/mark/libffi-sys-rs/build/common.rs:9:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
make: *** [build-capi-cranelift] Error 101

I've been getting these errors with libffi since I've started working off of a (faster) external harddrive on OSX. No amount of giving the relevant tools full filesystem access or disabling OSX's protections has made it work, but passing the system feature to libffi-rs has been a workaround for the time being. However I tried to set up both libffi-rs master and your fork/branch of libffi-sys-rs to try it out and passing the system feature no longer fixes this issue.

Perhaps I set it up incorrectly because with the system feature it shouldn't be trying to compile anything...

---

here's how we use it:

libffi = { version = "0.9.1-alpha.0", path = "/Users/mark/libffi-rs", feature = ["system"] }

and here's how I made libffi-rs use the correct version of libffi-sys-rs:

libffi-sys = { version = "0.9.2-alpha.0", path = "../libffi-sys-rs" }

All the paths are correct. No idea what could be causing this. Also if either of you happen to know how to fix compiling the C code base of libffi on an external harddrive with OSX, I'd be extremely appreciative 😆

[yorickpeterse]

@MarkMcCaskey Not sure if you specifying an explicit version has anything to do with it, but this is how you should be able to get things working:

[patch.crates-io]
libffi-sys = { git = "https://github.com/YorickPeterse/libffi-sys-rs.git", branch = "no-bindgen-patch" }
libffi = { git = "https://github.com/YorickPeterse/libffi-rs.git", branch = "no-bindgen-patch" }

This approach does not require messing with any versions or whatsoever.

What causes the error I'm not sure, but make sure that you have both automake and autoconf installed; not sure if any specific versions are required either.