Comments (10)
It's hard to investigate without a working environment, do you have a list of steps to reproduce your build issue locally using docker?
from setuptools-rust.
@messense Yes indeed I do, and let me know if you need anything else:
Clone my copy of our spksrc framework repository:
$ git clone https://github.com/th0ma7/spksrc.git
$ cd spksrc
$ git checkout rustc-wheels
Create the updated docker image:
$ docker build - < Dockerfile
...
Successfully built 882a0451f1bb
Access the docker image:
$ docker run -it -v $(pwd):/spksrc 882a0451f1bb /bin/bash
Build a python310
package that will invoke building cryptography==38.0.1
root@2ff602f23b71:/spksrc# cd spk/python310
root@2ff602f23b71:/spksrc/spk/python310/# make arch-armv7-7.0
Options of interest for building are:
make arch-aarch64-7.0
make arch-x64-7.0
make arch-evansport-7.0
To build for every archs and every support Synology DSM Linux version (e.g. 6.1 & 7.0): make all-supported
Note that build logs are being output on the console but are also available in build-<arch>-<DSM-version>.log
from setuptools-rust.
Follow to this, my issue seems to be somewhat due to setuptools-rust
not behaving properly using a crossenv
.
Here's my finding using docker image above (always using a x86_64
host) and also reproduced on a Debian 11 VM. My first thing to try is, is it even cross-compiling using rust? Lets take ripgrep
as an easy app to build:
# CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="/spksrc/toolchain/syno-aarch64-7.0/work/aarch64-unknown-linux-gnu/bin/aarch64-unknown-linux-gnu-gcc" CARGO_HOME="/opt/cargo" RUSTUP_HOME="/opt/rustup" cargo install ripgrep --target=aarch64-unknown-linux-gnu --force
# file rg
rg: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
It works! Now let's try that using CARGO_BUILD_TARGET=aarch64-unknown-linux-gnu
instead of enforcing --target
:
# CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="/spksrc/toolchain/syno-aarch64-7.0/work/aarch64-unknown-linux-gnu/bin/aarch64-unknown-linux-gnu-gcc" CARGO_BUILD_TARGET=aarch64-unknown-linux-gnu CARGO_HOME="/opt/cargo" RUSTUP_HOME="/opt/rustup" cargo install ripgrep --force
# file rg
rg: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
Still working! So the theory is that I have a working rustc
along with a proper recognition of my target gcc
as default linker and CARGO*
and RUSTUP*
environment variables are also recognized and functional. That's a good start :)
More on this first testing can be found SynoCommunity/spksrc#5435 (comment)
Then I've resumed at trying to build cryptography
using our spksrc
framework. It's rather easy to trigger with PR above, once the build is complete (e.g. make arch-armv7-7.0
) you simply have to first setup your PATH
to point towards the crossenv
first:
$ export PATH=/spksrc/spk/python310/work-aarch64-7.0/crossenv/cross/bin:/spksrc/spk/python310/work-aarch64-7.0/crossenv/cross/bin:$PATH
Then build cryptography
from the extracted source:
$ CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="/spksrc/toolchain/syno-aarch64-7.0/work/aarch64-unknown-linux-gnu/bin/aarch64-unknown-linux-gnu-gcc" \
CARGO_BUILD_TARGET=aarch64-unknown-linux-gnu \
CARGO_HOME="/opt/cargo" RUSTUP_HOME="/opt/rustup" \
PYO3_CROSS_LIB_DIR=/spksrc/spk/python310/work-aarch64-7.0/install/var/packages/python310/target/lib/ \
PYO3_CROSS_INCLUDE_DIR=/spksrc/spk/python310/work-aarch64-7.0/install/var/packages/python310/target/include/ \
python3 setup.py bdist_wheel --py-limited-api=cp36
Interestingly enough it gets a little farther using RUSTFLAGS
such as:
RUSTFLAGS="--target=aarch64-unknown-linux-gnu -Clinker=/spksrc/toolchain/syno-aarch64-7.0/work/aarch64-unknown-linux-gnu/bin/aarch64-unknown-linux-gnu-gcc" \
CARGO_HOME="/opt/cargo" RUSTUP_HOME="/opt/rustup" \
PYO3_CROSS_LIB_DIR=/spksrc/spk/python310/work-aarch64-7.0/install/var/packages/python310/target/lib/ \
PYO3_CROSS_INCLUDE_DIR=/spksrc/spk/python310/work-aarch64-7.0/install/var/packages/python310/target/include/ \
python3 setup.py bdist_wheel --py-limited-api=cp36
A call using crossenv pip
will also result with the same outcome:
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="/spksrc/toolchain/syno-aarch64-7.0/work/aarch64-unknown-linux-gnu/bin/aarch64-unknown-linux-gnu-gcc" \
CARGO_BUILD_TARGET=aarch64-unknown-linux-gnu \
CARGO_HOME="/opt/cargo" RUSTUP_HOME="/opt/rustup" \
PYO3_CROSS_LIB_DIR=/spksrc/spk/python310/work-aarch64-7.0/install/var/packages/python310/target/lib/ \
PYO3_CROSS_INCLUDE_DIR=/spksrc/spk/python310/work-aarch64-7.0/install/var/packages/python310/target/include/ \
pip install cryptography==38.0.1
My conclusion is that setuptools-rust
is somehow mixing-up things between host & target and not always using the proper linker, often returning to /usr/bin/ld
instead of the provided one.
from setuptools-rust.
# CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="/spksrc/toolchain/syno-aarch64-7.0/work/aarch64-unknown-linux-gnu/bin/aarch64-unknown-linux-gnu-gcc" CARGO_HOME="/opt/cargo" RUSTUP_HOME="/opt/rustup" cargo install ripgrep --target=aarch64-unknown-linux-gnu --force
# file rg
rg: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
Does it still work if you also add all the CFLAGS
/CXXFLAGS
/LDFLAGS
env vars?
from setuptools-rust.
Looks like this circles back to an issue I posted before? benfogle/crossenv#65
from setuptools-rust.
In our CI we cleared the LIBRARY_PATH
crossenv set via --env LIBRARY_PATH=
setuptools-rust/.github/workflows/ci.yml
Line 221 in 96684af
from setuptools-rust.
I confirm, editing the crossenv/cross/bin/python3
and removing the LIBRARY_PATH
from the extra_envs
variable works!!!!
I got a few SSL deprecation warnings, mays have to find a way to pass -Wno-unused-function
... But it works:
# ll dist/cryptography-38.0.1-cp36-abi3-linux_aarch64.whl
-rw-r--r-- 1 root root 1810579 Oct 1 08:41 dist/cryptography-38.0.1-cp36-abi3-linux_aarch64.whl
from setuptools-rust.
@messense is there an intent to include the --env LIBRARY_PATH=
as built-in to setuptools-rust
or should I apply a workaround on my end?
from setuptools-rust.
is there an intent to include the
--env LIBRARY_PATH=
as built-in tosetuptools-rust
No, we don't want to add too much assumption in setuptools-rust
, bad assumption caused a trouble in the past. It's better to be dealt by user.
Closing as resolved.
from setuptools-rust.
@messense having a global option similar to --library-path=
or --disable-library-path
would be useful and allow not breaking the code while adding extra functionality
from setuptools-rust.
Related Issues (20)
- build fails with "error: Rust build failed; unable to find any build artifacts" HOT 6
- Is there a way to trigger `release` target build for `build_ext --inplace` command? HOT 3
- Failed to build on MacOs 13.0.1,setuptools_rust HOT 1
- can I build an `.so` without the target between the module name and the extension? HOT 2
- Setuptools deprecation warning with setuptools-rust HOT 4
- Facing an issue while trying to build setuptools-rust HOT 3
- Compilation in setuptools_rust fails with `1.70.0-nightly` toolchain HOT 3
- Python crashes on import when using pip editable install on M1 HOT 6
- Unable to control `--debug` / `--release` via PEP517 backend HOT 7
- No module named 'setuptools.command.build' HOT 2
- Document how to use setuptools-rust with a workspace HOT 2
- Scaling PyO3 down with single-file Python extensions HOT 7
- DeprecationWarning: pkg_resources is deprecated as an API
- Allow offline mode via environment variable HOT 7
- `install_extension` crashes due to attempt to mkdir an empty path (v1.8.0)
- With latest nightly rust, builds failing due to changes in cargo output HOT 6
- Trouble using pyproject.toml for limited api HOT 8
- Please update PyO3 to 0.20.3 HOT 2
- test_metadata_cargo_log fails when run with Python 3.13.0b1 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from setuptools-rust.