benfogle / crossenv Goto Github PK
View Code? Open in Web Editor NEWCross-compiling virtualenv for Python
License: MIT License
Cross-compiling virtualenv for Python
License: MIT License
My setup.py conditionally runs an executable -- when cross compiling, it turns it off. I was looking for _PYTHON_HOST_PLATFORM
, but crossenv deletes that?
Now I'm checking for PYTHON_CROSSENV... but I'd be happier if it were a documented way to detect crossenv.
I'm unable to build matplotlib with crossenv. A project that demonstrates the problem is here.
After compiling host and build python and setting up the crossenv venv, the demo does:
ADD requirements.txt /
RUN $CROSS_VENV/bin/build-pip install -r requirements.txt
RUN $CROSS_VENV/bin/cross-pip install -r requirements.txt
where requirements.txt is:
cython==0.29.23
numpy==1.19.5
matplotlib==3.3.3
The cross build falls down in matplotlib, seemingly due to cython. The error is very long, so I've attached it to this issue (here).
cross-env
version: 0.7Relevant code or config:
With associated cross-py.zip file:
This will download an android-ndk, openssl & libffi, build the two libs & then download & build a Linux x64 Python 3.8.3 and an aarch64 Python 3.8.3. It then installs crossenv & creates the venv and uses it to build a numpy wheel for aarch64. All are built for Android API level 29.
The result is an install-tar.tgz file, which should be copied to a target platform. I've used both a Qualcom board running Android 10, as well as a Pixel 4 running Android 10 (both API level 29). I've tried connecting directly to both using adb & direct connect, but have had better luck (better tool support) using Termux.
Once on the android box, do:
tar -xf install-arm.tgz
cd install-arm
source runtime-env.sh # will set paths & install pip
bin/pip3 install ../numpy-1.18.4-cp38-cp38-linux_aarch64.whl
bin/python3 -m numpy
What happened:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/data/data/com.termux/files/home/install-arm/lib/python3.8/site-packages/numpy/__init__.py", line 142, in <module>
from . import core
File "/data/data/com.termux/files/home/install-arm/lib/python3.8/site-packages/numpy/core/__init__.py", line 100, in <module>
from . import _add_newdocs
File "/data/data/com.termux/files/home/install-arm/lib/python3.8/site-packages/numpy/core/_add_newdocs.py", line 4441, in <module>
add_newdoc('numpy.core._multiarray_tests', 'format_float_OSprintf_g',
File "/data/data/com.termux/files/home/install-arm/lib/python3.8/site-packages/numpy/core/function_base.py", line 506, in add_newdoc
new = getattr(__import__(place, globals(), {}, [obj]), obj)
ImportError: dlopen failed: cannot locate symbol "tanh" referenced by "/data/data/com.termux/files/home/install-arm/lib/python3.8/site-packages/numpy/core/_multiarray_tests.cpython-38.so"...
Reproduction repository:
See attached zip
Problem description:
Note that the cross-compiled Python executable works fine; pure-python packages can be installed and imported without issue (although occasionally some dependencies need to be installed). The error in question above relates to a function that lives in libm.so, which is in my path, and is also being loaded by python immediately on startup (strace shows me this).
Since working with Termux is an option, the fact that I can't build numpy is not a problem, per se, because I can just "pip install numpy" directly, and the version that is installed in this manner works correctly.
However, what I'm really trying to get working is opencv, and that cannot be installed via the pip repositories. After much effort, I've finally coaxed the opencv/python build systems into producing an aarch64 wheel, but unfortunately, I'm back to the same error (although in this case, it's complaining about a function in liblog.so instead of libm.so). The .so being generated appears to be correct, it's just that it's in an odd state that prevents the python/cython import loader from doing the right thing.
The opencv build is a lot more complicated, and takes very long compared to numpy; if we can come up with a solution that fixes numpy's problem, I can apply it to the opencv build.
Suggested solution:
I'm at my wit's end. As far as I can tell, I'm doing everything "right" but somehow the wheels I'm generating are subtly different from the ones that are in the aarch64 pip repositories.
Build-Python: /opt/python/cp310-cp310/bin/python
Cross-Python: /opt/_internal/xc/xcpython-3.10.0b3/bin/python3
Error encountered: AssertionError: sysconfig was set up prior to patching?
Logs:
[root:/work] # /opt/python/cp310-cp310/bin/python -m crossenv /opt/_internal/xc/xcpython-3.10.0b3/bin/python3 env
WARNING: Unable to symlink '/opt/python/cp310-cp310/bin/python' to '/work/env/cross/bin/python'
Traceback (most recent call last):
File "/opt/python/cp310-cp310/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/python/cp310-cp310/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/work/env/build/lib/python3.10/site-packages/pip/main.py", line 29, in
from pip._internal.cli.main import main as _main
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 9, in
from pip._internal.cli.autocompletion import autocomplete
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/cli/autocompletion.py", line 10, in
from pip._internal.cli.main_parser import create_main_parser
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/cli/main_parser.py", line 8, in
from pip._internal.cli import cmdoptions
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/cli/cmdoptions.py", line 23, in
from pip._internal.cli.parser import ConfigOptionParser
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/cli/parser.py", line 12, in
from pip._internal.configuration import Configuration, ConfigurationError
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/configuration.py", line 27, in
from pip._internal.utils.misc import ensure_dir, enum
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 42, in
from pip._internal.locations import get_major_minor_version, site_packages, user_site
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/locations/init.py", line 9, in
from . import _distutils, _sysconfig
File "/work/env/build/lib/python3.10/site-packages/pip/_internal/locations/_distutils.py", line 9, in
from distutils.command.install import SCHEME_KEYS
File "/opt/python/cp310-cp310/lib/python3.10/distutils/command/install.py", line 13, in
from distutils.sysconfig import get_config_vars
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "/work/env/lib/site.py", line 68, in exec_module
_patch_module(module, self.patch)
File "/work/env/lib/site.py", line 55, in _patch_module
exec(src, module.dict, module.dict)
File "", line 21, in
AssertionError: sysconfig was set up prior to patching?
ERROR: Command '['/work/env/cross/bin/python', '-m', 'pip', '--disable-pip-version-check', 'install', '--ignore-installed', '--prefix=/work/env/cross', 'pip==21.1.1', 'setuptools==56.0.0']' returned non-zero exit status 1.
[root:/work] 3s 1 #
See #73
os.uname().machine
will never report "arm", but something like "armv7l". There are several options, and we have no way of knowing which one will be correct. We should still pick one as a sane default, now that we can override the value if needed.
Hello, I'm trying to cross-compile NumPy 1.17.3 for armhf (as a dependency for pandas) on an x86-64 computer running Ubuntu 20.04. Both host and build Python are version 3.8.5, built from source. I've succeeded in creating the environment, but the compilation fails with the following message:
gcc -pthread -shared build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o -Lbuild/temp.l--x86_64-3.8 -lm -o build/lib.l--x86_64-3.8/numpy/core/_dummy.cpython-38-arm-linux-gnueabihf.so
/usr/bin/ld: build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o: relocations in generic ELF (EM: 40)
/usr/bin/ld: build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o: relocations in generic ELF (EM: 40)
/usr/bin/ld: build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o: relocations in generic ELF (EM: 40)
/usr/bin/ld: build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o: relocations in generic ELF (EM: 40)
/usr/bin/ld: build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o: relocations in generic ELF (EM: 40)
/usr/bin/ld: build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o: relocations in generic ELF (EM: 40)
/usr/bin/ld: build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
error: Command "gcc -pthread -shared build/temp.l--x86_64-3.8/numpy/core/src/dummymodule.o -Lbuild/temp.l--x86_64-3.8 -lm -o build/lib.l--x86_64-3.8/numpy/core/_dummy.cpython-38-arm-linux-gnueabihf.so" failed with exit status 1
Looking at the output, I think that for some reason, it seems to want to link for x86-64 using /usr/bin/ld despite the binaries being compiled for armhf. I have the following environment variables set:
CC=arm-linux-gnueabihf-gcc
CXX=arm-linux-gnueabihf-g++
LD=arm-linux-gnueabihf-ld
Hello,
I have created the venv with this command:
/usr/local/bin/python3 -m crossenv /opt/buildroot-2020.08/output/target/usr/bin/python3 venv
Then I have launched:
. venv/bin/activate
build-pip install websockets
pip -v install websockets
Using pip 21.3.1 from /home/sylvain/venv/build/lib/python3.8/site-packages/pip (python 3.8)
Collecting websockets
Using cached websockets-10.1.tar.gz (83 kB)
Running command python setup.py egg_info
running egg_info
creating /tmp/pip-pip-egg-info-cbb524bm/websockets.egg-info
writing /tmp/pip-pip-egg-info-cbb524bm/websockets.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-cbb524bm/websockets.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-pip-egg-info-cbb524bm/websockets.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-cbb524bm/websockets.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-pip-egg-info-cbb524bm/websockets.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file '/tmp/pip-pip-egg-info-cbb524bm/websockets.egg-info/SOURCES.txt'
Preparing metadata (setup.py) ... done
Building wheels for collected packages: websockets
Running command /home/sylvain/venv/cross/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-25q9colm/websockets_727dd66a3ea14744813329ee6422f3a6/setup.py'"'"'; file='"'"'/tmp/pip-install-25q9colm/websockets_727dd66a3ea14744813329ee6422f3a6/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-63mn25_z
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.8
creating build/lib.linux-aarch64-3.8/websockets
copying src/websockets/init.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/streams.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/http.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/exceptions.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/utils.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/main.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/datastructures.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/imports.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/frames.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/typing.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/client.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/uri.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/headers.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/version.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/auth.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/server.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/connection.py -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/http11.py -> build/lib.linux-aarch64-3.8/websockets
creating build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/framing.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/init.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/http.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/handshake.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/client.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/compatibility.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/auth.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/protocol.py -> build/lib.linux-aarch64-3.8/websockets/legacy
copying src/websockets/legacy/server.py -> build/lib.linux-aarch64-3.8/websockets/legacy
creating build/lib.linux-aarch64-3.8/websockets/extensions
copying src/websockets/extensions/permessage_deflate.py -> build/lib.linux-aarch64-3.8/websockets/extensions
copying src/websockets/extensions/init.py -> build/lib.linux-aarch64-3.8/websockets/extensions
copying src/websockets/extensions/base.py -> build/lib.linux-aarch64-3.8/websockets/extensions
running egg_info
writing src/websockets.egg-info/PKG-INFO
writing dependency_links to src/websockets.egg-info/dependency_links.txt
writing top-level names to src/websockets.egg-info/top_level.txt
reading manifest file 'src/websockets.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/websockets.egg-info/SOURCES.txt'
copying src/websockets/py.typed -> build/lib.linux-aarch64-3.8/websockets
copying src/websockets/speedups.c -> build/lib.linux-aarch64-3.8/websockets
running build_ext
building 'websockets.speedups' extension
creating build/temp.linux-aarch64-3.8
creating build/temp.linux-aarch64-3.8/src
creating build/temp.linux-aarch64-3.8/src/websockets
/opt/buildroot-2020.08/output/host/bin/aarch64-buildroot-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fPIC -I/home/sylvain/venv/cross/include -I/opt/buildroot-2020.08/output/target/usr/include/python3.8 -c src/websockets/speedups.c -o build/temp.linux-aarch64-3.8/src/websockets/speedups.o
src/websockets/speedups.c:4:10: fatal error: Python.h: No such file or directory
4 | #include <Python.h>
| ^~~~~~~~~~
compilation terminated.
warning: build_ext: building extension "websockets.speedups" failed: command '/opt/buildroot-2020.08/output/host/bin/aarch64-buildroot-linux-gnu-gcc' failed with exit status 1
How can I fix this issue since Python.h exists in the sysroot?
find . -iname 'Python.h'
./buildroot-2020.08/output/build/host-python-2.7.18/Include/Python.h
./buildroot-2020.08/output/build/buildroot-config/br2/target/uboot/needs/python.h
./buildroot-2020.08/output/build/host-python3-3.8.5/Include/Python.h
./buildroot-2020.08/output/build/python3-3.8.5/Include/Python.h
./buildroot-2020.08/output/build/host-gdb-8.3.1/gdb/python/python.h
./buildroot-2020.08/output/build/gdb-8.3.1/gdb/python/python.h
./buildroot-2020.08/output/host/include/python3.8/Python.h
./buildroot-2020.08/output/host/include/python2.7/Python.h
./buildroot-2020.08/output/host/aarch64-buildroot-linux-gnu/sysroot/usr/include/python3.8/Python.h
Note that the same error is also thrown for others modules:
pip -v install cffi
Using pip 21.3.1 from /home/sylvain/venv/build/lib/python3.8/site-packages/pip (python 3.8)
Collecting cffi
Using cached cffi-1.15.0.tar.gz (484 kB)
Running command python setup.py egg_info
running egg_info
creating /tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info
writing /tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/dependency_links.txt
writing entry points to /tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/entry_points.txt
writing requirements to /tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/requires.txt
writing top-level names to /tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file '/tmp/pip-pip-egg-info-uyy0zky7/cffi.egg-info/SOURCES.txt'
Preparing metadata (setup.py) ... done
Requirement already satisfied: pycparser in /home/sylvain/venv/cross/lib/python3.8/site-packages (from cffi) (2.21)
Building wheels for collected packages: cffi
Running command /home/sylvain/venv/cross/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3ceq5uqy/cffi_05b7b531340c4fc8ab6cacb1d2a6d399/setup.py'"'"'; file='"'"'/tmp/pip-install-3ceq5uqy/cffi_05b7b531340c4fc8ab6cacb1d2a6d399/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ryuc2qh1
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.8
creating build/lib.linux-aarch64-3.8/cffi
copying cffi/init.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/api.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/ffiplatform.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/commontypes.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/lock.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/cparser.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/vengine_gen.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/error.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/verifier.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/pkgconfig.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/recompiler.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/model.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/_cffi_include.h -> build/lib.linux-aarch64-3.8/cffi
copying cffi/parse_c_type.h -> build/lib.linux-aarch64-3.8/cffi
copying cffi/_embedding.h -> build/lib.linux-aarch64-3.8/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-aarch64-3.8/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-aarch64-3.8
creating build/temp.linux-aarch64-3.8/c
/opt/buildroot-2020.08/output/host/bin/aarch64-buildroot-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/home/sylvain/venv/cross/include -I/opt/buildroot-2020.08/output/target/usr/include/python3.8 -c c/_cffi_backend.c -o build/temp.linux-aarch64-3.8/c/_cffi_backend.o
c/_cffi_backend.c:2:10: fatal error: Python.h: No such file or directory
Regards
After installing crossenv (sudo pip3.5 install crossenv
) I tried to create the virtual environment but end up with:
python3.5 -m crossenv ~/python-host-3.5/usr/bin/python3 cross_venv [11:39:43][1]
WARNING: CC is a compound command (arm-tdx-linux-gnueabi-gcc -march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -DNDEBUG -fno-inline -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security )
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CC='arm-tdx-linux-gnueabi-gcc' and CFLAGS='-march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -DNDEBUG -fno-inline -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security'
WARNING: CXX is a compound command (arm-tdx-linux-gnueabi-g++ -march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -DNDEBUG -fno-inline -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security )
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CXX='arm-tdx-linux-gnueabi-g++' and CXXFLAGS='-march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -DNDEBUG -fno-inline -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security'
ERROR: Cannot find Makefile
I understand the warnings (but tend to ignore them for now), but I do not understand the error. Which Makefile is crossenv looking for?
I'm trying to cross compile Scipy, and I think I'm getting close. I've succesfully cross compiled its dependencies, OpenBLAS and Numpy, and am trying to cross compile Scipy itself with:
F90=$(CROSS_FC) \
F77=$(CROSS_FC) \
AR=$(CROSS_AR) \
RANLIB=$(CROSS_RANLIB) \
LD=$(CROSS_LD) \
cross-python setup.py build_ext \
bdist_wheel
Where $(CROSS_XX) is an appropriate toolchain.
The build almost finishes. It is currently failing in the linking stage with:
arm-linux-gnueabihf-gcc -shared -Wl,-s -L/home/abemtk/apps/python-cross-compiling-example/working/python_armhf/lib -L//usr/lib build/temp.l--x-3.6/scipy/spatial/src/distance_wrap.o -L/home/abemtk/apps/python-cross-compiling-example/working/crossenv/build/lib/python3.6/site-packages/numpy/core/lib -L/home/abemtk/apps/python-cross-compiling-example/working/python_armhf/lib -Lbuild/temp.l--x-3.6 -lnpymath -lm -lpython3.6m -lm -o build/lib.l--x-3.6/scipy/spatial/_distance_wrap.cpython-36m-arm-linux-gnueabihf.so -Wl,--version-script=build/temp.l--x-3.6/link-version-scipy.spatial._distance_wrap.map
/home/abemtk/apps/python-cross-compiling-example/working/crossenv/build/lib/python3.6/site-packages/numpy/core/lib/libnpymath.a: error adding symbols: File format not recognized
collect2: error: ld returned 1 exit status
error: Command "arm-linux-gnueabihf-gcc -shared -Wl,-s -L/home/abemtk/apps/python-cross-compiling-example/working/python_armhf/lib -L//usr/lib build/temp.l--x-3.6/scipy/spatial/src/distance_wrap.o -L/home/abemtk/apps/python-cross-compiling-example/working/crossenv/build/lib/python3.6/site-packages/numpy/core/lib -L/home/abemtk/apps/python-cross-compiling-example/working/python_armhf/lib -Lbuild/temp.l--x-3.6 -lnpymath -lm -lpython3.6m -lm -o build/lib.l--x-3.6/scipy/spatial/_distance_wrap.cpython-36m-arm-linux-gnueabihf.so -Wl,--version-script=build/temp.l--x-3.6/link-version-scipy.spatial._distance_wrap.map" failed with exit status 1
It looks like it is trying to statically link one of Numpy's libraries, but it's trying to do so against Numpy installed in build-python, which obviously doesn't work. Host-numpy is installed in /home/abemtk/apps/python-cross-compiling-example/working/python_armhf/lib
, which is the first search path, so I'm a little surprised it doesn't link to that instead. Edit: I think it needs to look specifically in lib/python3.6/site-packages/numpy/core/lib
. I tried adding that to Scipy's site.cfg, but no dice.
I've tried uninstalling Numpy from build-python, but that just results in a ModuleNotFoundError.
Host-python may use a compiler that the user doesn't expect. This may be because it was compiled natively on the host (see #24), PATH being set up incorrectly, typos, and so on. We could provide a warning in the following circumstances:
I'm getting this error message when trying to create the virtual environment:
Failed to import the site module
Traceback (most recent call last):
File "/home/alexander/apps/python-cross-compiling-example/working/crossenv/lib/site.py", line 53, in <module>
abiflags = sysconfig.get_config_var('ABIFLAGS')
File "/home/alexander/apps/python-cross-compiling-example/working/build-python/install/lib/python3.6/sysconfig.py", line 601, in get_config_var
return get_config_vars().get(name)
File "/home/alexander/apps/python-cross-compiling-example/working/build-python/install/lib/python3.6/sysconfig.py", line 550, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/home/alexander/apps/python-cross-compiling-example/working/build-python/install/lib/python3.6/sysconfig.py", line 421, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata_m_linux_.cpython-36'; '_sysconfigdata_m_linux_' is not a package
A module by that name does exist in host-python, but in build-python it is called _sysconfigdata_m_linux_x86_64-linux-gnu. I'm not sure which one is supposed to be imported. A comment in crossenv/lib/site.py seems to reference this problem:
# sysconfig isn't quite set up right, because it queries from a not-yet fixed
# sys module. The variables that come from build_time_vars are correct, so
# we can safely use those. We'll re-import it later once sys is fixed.
By the way, as you can see, here I am using the example from https://github.com/benfogle/python-cross-compiling-example, but I have the same issue outside that environment.
Edit: have changed the title to be relevant to the solution in case anyone stumbles across the same limitation at a later point
Im grasping at straws here, so really not sure if you would be able to help with a failure to setup crossenv for aarch64-linux-gnu host on an x86_64-linux-gnu build.
Ive been implementing the usage of crossenv to tidy up the build system for the Kodi media centre. In the past they have been patching wildly to get modules like pillow and pycrptodome to build for the myriad of platforms (x86_64/aarch64/armv7 for android/apple/linux).
Ive implemented it, and its working great for all our build systems except our aarch64-linux target on our build server (utilising Jenkins).
We have a build python (native in our parlance) that we build from scratch for py3.8.5 (also have 3.8.8 as a test bed).
The host python (target) is again py3.8.5 (or 3.8.8 as is appropriate). All dependencies are built from source except the compiler toolchain.
The build command from the build server log for crossenv is below
cd aarch64-linux-gnu-debug; /home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/bin/python3 -m crossenv \
\
--env=CPATH:=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/include \
--env=LIBRARY_PATH:=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/lib \
--env=PATH:=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/bin \
--env=CFLAGS:="-fPIC -DPIC -Og -g -D_DEBUG -isystem /home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/include" \
--env=CXXFLAGS:="-fPIC -DPIC -Og -g -D_DEBUG -isystem /home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/include" \
--env=LDFLAGS:="-L/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/lib -Wl,-rpath-link=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/lib " \
\
--env=PKG_CONFIG:=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/bin/pkg-config \
--env=PKG_CONFIG_PATH:=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/lib/pkgconfig \
--env=PKG_CONFIG_LIBDIR:=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/lib/pkgconfig \
\
--env=PYTHONPATH:=/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/lib/python3.8/site-packages \
\
--cc "/usr/bin/ccache /usr/bin/aarch64-linux-gnu-gcc " \
--cxx "/usr/bin/ccache /usr/bin/aarch64-linux-gnu-g++" \
--ar "/usr/bin/aarch64-linux-gnu-ar" \
-vv \
/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/aarch64-linux-gnu-debug/bin/python3 \
/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/bin/koditargetenv
The verbose failure we get is
08:25:50 WARNING: CC is a compound command (['/usr/bin/ccache', '/usr/bin/aarch64-linux-gnu-gcc'])
08:25:50 WARNING: This can cause issues for modules that don't expect it.
08:25:50 WARNING: Consider setting CC='/usr/bin/ccache' and CFLAGS='/usr/bin/aarch64-linux-gnu-gcc'
08:25:50 WARNING: CXX is a compound command (['/usr/bin/ccache', '/usr/bin/aarch64-linux-gnu-g++'])
08:25:50 WARNING: This can cause issues for modules that don't expect it.
08:25:50 WARNING: Consider setting CXX='/usr/bin/ccache' and CXXFLAGS='/usr/bin/aarch64-linux-gnu-g++'
08:25:50 INFO: Creating build-python environment
08:26:28 INFO: Creating cross-python environment
08:26:28 INFO: Installing cross-pip
08:26:28 DEBUG: Installing: ['pip==20.2.3', 'setuptools==49.2.1']
08:26:28 ERROR: [Errno 8] Exec format error: '/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/bin/koditargetenv/cross/bin/python3'
08:26:28 DEBUG: Traceback:
08:26:28 Traceback (most recent call last):
08:26:28 File "/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/target/pythonmodule-crossenv/aarch64-linux-gnu-debug/crossenv/__init__.py", line 938, in main
08:26:28 builder.create(env_dir)
08:26:28 File "/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/target/pythonmodule-crossenv/aarch64-linux-gnu-debug/crossenv/__init__.py", line 414, in create
08:26:28 self.make_cross_python(context)
08:26:28 File "/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/target/pythonmodule-crossenv/aarch64-linux-gnu-debug/crossenv/__init__.py", line 670, in make_cross_python
08:26:28 subprocess.check_output([context.cross_env_exe, '-m', 'pip',
08:26:28 File "/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/lib/python3.8/subprocess.py", line 415, in check_output
08:26:28 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
08:26:28 File "/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/lib/python3.8/subprocess.py", line 493, in run
08:26:28 with Popen(*popenargs, **kwargs) as process:
08:26:28 File "/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/lib/python3.8/subprocess.py", line 858, in __init__
08:26:28 self._execute_child(args, executable, preexec_fn, close_fds,
08:26:28 File "/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/lib/python3.8/subprocess.py", line 1706, in _execute_child
08:26:28 raise child_exception_type(errno_num, err_msg, err_filename)
08:26:28 OSError: [Errno 8] Exec format error: '/home/jenkins/workspace/LINUX-AARCH64-GLES/tools/depends/xbmc-depends/x86_64-linux-gnu-native/bin/koditargetenv/cross/bin/python3'
The kicker for me, i cant replicate it locally on a VM. I do the apple dev side of things mostly for kodi, so ive been doing android and linux build testing in a vm.
The build host python3 when built works fine when called. the crossenv build works fully for android x86_64, aarch64, armv7 as well as for Apple ios/tvos aarch64 and OSX x86_64. Our jenkins builds even work fine for Linux x86_64 and armv7
scratching my head here and really grasping at straws.
Hoping you might have some thoughts that get me on the right track to get it resolved.
Test for things like platform.uname()
, etc.
I've some problem to cross compile a package that uses the cffi dependency.
Here is a part of my build script:
# install cffi for build env as recommended
build-pip install cffi
# just in case, I don't really know if it's usefull in this case
cross-expose cffi
# package that uses cryptography and so cffi
pip install azure-storage-blob==12.5.0
And the output in the console:
WARNING: CC is a compound command (['arm-gad-linux-gnueabi-gcc', '-march=armv7-a', '-mthumb-interwork', '-mfloat-abi=hard', '-mfpu=neon', '-mtune=cortex-a9', '--sysroot=/opt/gad/1.0.0/sysroots/cortexa9hf-vfp-neon-gad-linux-gnueabi'])
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CC='arm-gad-linux-gnueabi-gcc' and CFLAGS='-march=armv7-a -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/opt/gad/1.0.0/sysroots/cortexa9hf-vfp-neon-gad-linux-gnueabi'
WARNING: CXX is a compound command (['arm-gad-linux-gnueabi-g++', '-march=armv7-a', '-mthumb-interwork', '-mfloat-abi=hard', '-mfpu=neon', '-mtune=cortex-a9', '--sysroot=/opt/gad/1.0.0/sysroots/cortexa9hf-vfp-neon-gad-linux-gnueabi'])
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CXX='arm-gad-linux-gnueabi-g++' and CXXFLAGS='-march=armv7-a -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/opt/gad/1.0.0/sysroots/cortexa9hf-vfp-neon-gad-linux-gnueabi'
Collecting cffi
Downloading https://files.pythonhosted.org/packages/50/ca/bbca0fd95b24a1d4f0d2e016f09f35ae68d4fe72bf34cc538d0a0d2d3e10/cffi-1.14.3-cp36-cp36m-manylinux1_x86_64.whl (400kB)
Collecting pycparser (from cffi)
Downloading https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024debf82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl (112kB)
Installing collected packages: pycparser, cffi
Successfully installed cffi-1.14.3 pycparser-2.20
You are using pip version 18.1, however version 20.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting azure-storage-blob==12.5.0
Downloading https://files.pythonhosted.org/packages/b8/84/7e51b3e1156bcb89a20b9ec641d4fced4800aa79daac3a403898c32046be/azure_storage_blob-12.5.0-py2.py3-none-any.whl (326kB)
Collecting azure-core<2.0.0,>=1.6.0 (from azure-storage-blob==12.5.0)
Downloading https://files.pythonhosted.org/packages/4c/fa/46974f4a7ad78b27e3eda8a573cc0c2508849f0d7d360b61c07cc5b46014/azure_core-1.8.2-py2.py3-none-any.whl (122kB)
Collecting msrest>=0.6.10 (from azure-storage-blob==12.5.0)
Downloading https://files.pythonhosted.org/packages/fa/f5/9e315fe8cb985b0ce052b34bcb767883dc739f46fadb62f05a7e6d6eedbe/msrest-0.6.19-py2.py3-none-any.whl (84kB)
Collecting cryptography>=2.1.4 (from azure-storage-blob==12.5.0)
Downloading https://files.pythonhosted.org/packages/5d/4b/7bb135c5787c003cdbc44990c5f41908f0f37135e0bb554e880d90fd5f6f/cryptography-3.1.1.tar.gz (534kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
Complete output from command /tmp/python-cross-env/cross/bin/python3.6 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-k4ijf2ws --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=40.6.0 wheel "cffi>=1.8,!=1.11.3; platform_python_implementation != 'PyPy'":
*******************************************************
* Crossenv has leaked into another Python interpreter!
* You should probably file a bug report.
* Version 3.6.8 (default, Dec 25 2018, 00:00:00)
[GCC 4.8.4]
* Executable /tmp/python-cross-env/cross/bin/python3.6
*******************************************************
Collecting setuptools>=40.6.0
Using cached https://files.pythonhosted.org/packages/6d/38/c21ef5034684ffc0412deefbb07d66678332290c14bb5269c85145fbd55e/setuptools-50.3.2-py3-none-any.whl
Collecting wheel
Using cached https://files.pythonhosted.org/packages/a7/00/3df031b3ecd5444d572141321537080b40c1c25e1caa3d86cdd12e5e919c/wheel-0.35.1-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8
Downloading https://files.pythonhosted.org/packages/cb/ae/380e33d621ae301770358eb11a896a34c34f30db188847a561e8e39ee866/cffi-1.14.3.tar.gz (470kB)
Collecting pycparser (from cffi!=1.11.3,>=1.8)
Using cached https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024debf82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl
Installing collected packages: setuptools, wheel, pycparser, cffi
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
Complete output from command /tmp/python-cross-env/cross/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-b363qank/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-dcmdpimn/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-k4ijf2ws --compile --install-headers /tmp/python-cross-env/cross/include/site/python3.6/cffi:
*******************************************************
* Crossenv has leaked into another Python interpreter!
* You should probably file a bug report.
* Version 3.6.8 (default, Dec 25 2018, 00:00:00)
[GCC 4.8.4]
* Executable /tmp/python-cross-env/cross/bin/python3.6
*******************************************************
*******************************************************
* Crossenv has leaked into another Python interpreter!
* You should probably file a bug report.
* Version 3.6.8 (default, Dec 25 2018, 00:00:00)
[GCC 4.8.4]
* Executable /tmp/python-cross-env/cross/bin/python3.6
*******************************************************
running install
running build
running build_py
creating build
creating build/lib.linux-arm-3.6
creating build/lib.linux-arm-3.6/cffi
copying cffi/api.py -> build/lib.linux-arm-3.6/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-arm-3.6/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-arm-3.6/cffi
copying cffi/ffiplatform.py -> build/lib.linux-arm-3.6/cffi
copying cffi/verifier.py -> build/lib.linux-arm-3.6/cffi
copying cffi/error.py -> build/lib.linux-arm-3.6/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-arm-3.6/cffi
copying cffi/model.py -> build/lib.linux-arm-3.6/cffi
copying cffi/recompiler.py -> build/lib.linux-arm-3.6/cffi
copying cffi/vengine_gen.py -> build/lib.linux-arm-3.6/cffi
copying cffi/__init__.py -> build/lib.linux-arm-3.6/cffi
copying cffi/cparser.py -> build/lib.linux-arm-3.6/cffi
copying cffi/lock.py -> build/lib.linux-arm-3.6/cffi
copying cffi/commontypes.py -> build/lib.linux-arm-3.6/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-arm-3.6/cffi
copying cffi/pkgconfig.py -> build/lib.linux-arm-3.6/cffi
copying cffi/_cffi_include.h -> build/lib.linux-arm-3.6/cffi
copying cffi/parse_c_type.h -> build/lib.linux-arm-3.6/cffi
copying cffi/_embedding.h -> build/lib.linux-arm-3.6/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-arm-3.6/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-arm-3.6
creating build/temp.linux-arm-3.6/c
arm-gad-linux-gnueabi-gcc -march=armv7-a -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/opt/gad/1.0.0/sysroots/cortexa9hf-vfp-neon-gad-linux-gnueabi -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/tmp/python-cross-env/cross/include -I/opt/gad/1.0.0/sysroots/cortexa9hf-vfp-neon-gad-linux-gnueabi/usr/include/python3.6m -c c/_cffi_backend.c -o build/temp.linux-arm-3.6/c/_cffi_backend.o
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^
compilation terminated.
error: command 'arm-gad-linux-gnueabi-gcc' failed with exit status 1
You can see that the pip command uses --ignore-installed probably because I use a Debian environment to build, but I don't know if it's relevant. I've tried to deactivate it without success.
Hello,
I can't build cryptography since Rust compiler isn't found.
Python: 3.8.5
platform: Linux-aarch64-with
pip: n/a
setuptools: 60.6.0
setuptools_rust: 1.1.2error: can't find Rust compiler
I have created the venv with this command:
/usr/local/bin/python3 -m crossenv /opt/buildroot-2020.08/output/host/aarch64-buildroot-linux-gnu/sysroot/usr/bin/python3 venv
All related rust stuff is here:
ls /opt/buildroot-2020.08/output/host/bin/*rust*
/opt/buildroot-2020.08/output/host/bin/rustc
/opt/buildroot-2020.08/output/host/bin/rust-gdb
/opt/buildroot-2020.08/output/host/bin/rust-lldb
/opt/buildroot-2020.08/output/host/bin/rustdoc
/opt/buildroot-2020.08/output/host/bin/rust-gdbgui
How can crossenv find rustc?
Regards
I can't even create the virtual environment and get "Failed to import site module".
This is crossenv 0.6 on opensuse leap 15.1. Target host is Raspberry Pi with a custom arm toolchain. Python on suse is 3.6.10 and 3.6.5 on target. Close enough?
/usr/bin/python3 -m crossenv /nentec/work/geulig/NENUX/nenux.git/user/python/build/python venv
WARNING: CC is a compound command (arm-pgc-linux-gnueabi-gcc -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include -O1 -pipe -fno-common -fno-builtin -Wall -Dlinux -D__linux__ -Dunix -DEMBED -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include)
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CC='arm-pgc-linux-gnueabi-gcc' and CFLAGS='-mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include -O1 -pipe -fno-common -fno-builtin -Wall -Dlinux -D__linux__ -Dunix -DEMBED -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include'
Failed to import the site module
Traceback (most recent call last):
File "/usr/lib64/python3.6/sysconfig.py", line 145, in _subst_vars
return s.format(**local_vars)
KeyError: 'platlibdir'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/sysconfig.py", line 148, in _subst_vars
return s.format(**os.environ)
KeyError: 'installed_base'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/lib/site.py", line 62, in <module>
stdlib = os.path.normpath(sysconfig.get_path('stdlib'))
File "/usr/lib64/python3.6/sysconfig.py", line 512, in get_path
return get_paths(scheme, vars, expand)[name]
File "/usr/lib64/python3.6/sysconfig.py", line 502, in get_paths
return _expand_vars(scheme, vars)
File "/usr/lib64/python3.6/sysconfig.py", line 169, in _expand_vars
res[key] = os.path.normpath(_subst_vars(value, vars))
File "/usr/lib64/python3.6/sysconfig.py", line 150, in _subst_vars
raise AttributeError('{%s}' % var)
AttributeError: {'installed_base'}
ERROR: Command '['/home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/cross/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'install', '--ignore-installed', '--prefix=/home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/cross', 'pip==18.1', 'setuptools==40.6.2']' returned non-zero exit status 1.
make[2]: *** [Makefile:61: venv] Fehler 1
I tried upgrading to 1.1.0, but activating venv fails now:
/bin/sh: /home/geulig/linux.git/user/python_modules/venv/cross/bin/pip3: /home/geulig/linux.git/user/python/build/bin/python: bad interpreter: No such file or directory
make[2]: *** [Makefile:40: all] Error 1
The shebang of venv/cross/bin/pip3 is
#!/home/geulig/linux.git/user/python/build/bin/python
and yes, this file does not exist.
Version 1.0 still works. Here the shebang is
#!/home/geulig/linux.git/user/python_modules/venv/cross/bin/python
Setup is done like this:
PYTHONHOME="/opt/python-3.6.13" \
PYTHONPATH="/opt/python-3.6.13/lib/python3.6/site-packages:/opt/python-3.6.13/lib64/python3.6/lib-dynload" \
/opt/python-3.6.13/bin/python3 -m crossenv /home/geulig/linux.git/user/python/build/python venv
. venv/bin/activate
Maybe this has something to do with the changes for #44/#46?
After creating the cross venv with sysroot set to the correct directory in Android-NDK, I ran those commands:
$ build-pip install cffi
$ pip install cryptography==2.8
I then moved the cross-compiled modules from the cross directory to the library of the cross compiled version of Python. When I run my Python program on Android, I get:
from cryptography.hazmat.bindings._constant_time import lib
File "<python>/importlib/_bootstrap.py", line 978, in _find_and_load
File "<python>/importlib/_bootstrap.py", line 967, in _find_and_load_unlocked
File "<python>/importlib/_bootstrap.py", line 666, in _load_unlocked
File "<python>/importlib/_bootstrap.py", line 577, in module_from_spec
File "<python>/importlib/_bootstrap_external.py", line 914, in create_module
File "<python>/importlib/_bootstrap.py", line 222, in _call_with_frames_removed
File "<python>/importlib/_bootstrap.py", line 978, in _find_and_load
File "<python>/importlib/_bootstrap.py", line 967, in _find_and_load_unlocked
File "<python>/importlib/_bootstrap.py", line 666, in _load_unlocked
File "<python>/importlib/_bootstrap.py", line 577, in module_from_spec
File "<python>/importlib/_bootstrap_external.py", line 914, in create_module
File "<python>/importlib/_bootstrap.py", line 222, in _call_with_frames_removed
ImportError: dlopen failed: library "libpthread.so.0" not found
I am on purpose using cryptography==2.8 since it does not contain the -lpthread LDFLAG. But it looks like the _constant_time.abi3.so file expects the libpthread.so.0 of the Build machine.
In reference to #44. It's a very specific condition, but not too hard to deal with on our end. If the shebang line is going to be too large, we can make a helper script (using /bin/sh
) that executes the real one directly as /full/path/to/build-python -I /full/path/to/cross-python
.
I'm probably using it wrong, but after installing numpy, scipy does not build:
$ cd scipy-1.1.0
$ python setup.py install
Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:
- `pip install .` (from a git repo or downloaded source
release)
- `pip install scipy` (last SciPy release on PyPI)
Traceback (most recent call last):
File "/home/ori/src/py/xcenv/cross/lib/python3.6/site-packages/numpy/core/__init__.py", line 16, in <module>
from . import multiarray
ImportError: cannot import name 'multiarray'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 474, in <module>
setup_package()
File "setup.py", line 450, in setup_package
from numpy.distutils.core import setup
File "/home/ori/src/py/xcenv/cross/lib/python3.6/site-packages/numpy/__init__.py", line 142, in <module>
from . import add_newdocs
File "/home/ori/src/py/xcenv/cross/lib/python3.6/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/home/ori/src/py/xcenv/cross/lib/python3.6/site-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/home/ori/src/py/xcenv/cross/lib/python3.6/site-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/home/ori/src/py/xcenv/cross/lib/python3.6/site-packages/numpy/core/__init__.py", line 26, in <module>
raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control). Otherwise reinstall numpy.
This seems to be the issue mentioned in the readme, where cross-expose needs to be run in order to get numpy to work, but cross-expose doesn't seem to find numpy:
cross-expose numpy
WARNING: 'numpy' was not found in build-python. Skipping.
Also, as mentioned in my other issue, pip doesn't work. It seems that this is also the case for cross-pip:
(cross) ; cross-pip numpy
Traceback (most recent call last):
File "/home/ori/src/py/xcenv/bin/cross-pip", line 7, in <module>
from pip._internal import main
ModuleNotFoundError: No module named 'pip._internal'
It seems that the environment doesn't get setup properly? I'm getting an error in my container:
root@8caac6e650ba:/build/venv/cross/bin# source activate
(cross) root@8caac6e650ba:/build/venv/cross/bin# python -m sysconfig
Platform: "l--x86_64"
Python version: "3.8"
Current installation scheme: "posix_prefix"
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.8/sysconfig.py", line 716, in <module>
_main()
File "/usr/lib/python3.8/sysconfig.py", line 710, in _main
_print_dict('Paths', get_paths())
File "/usr/lib/python3.8/sysconfig.py", line 502, in get_paths
return _expand_vars(scheme, vars)
File "/usr/lib/python3.8/sysconfig.py", line 172, in _expand_vars
_extend_dict(vars, get_config_vars())
File "/usr/lib/python3.8/sysconfig.py", line 550, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/usr/lib/python3.8/sysconfig.py", line 421, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata__arm-linux-gnueabi'
That file does exist in my cross python directory structure. Do you have any idea what might be missing? I'm happy to dig into it with a little direction...
I need to cross compile my project to armv7l
, I compiled python to armv7l
, but crossenv returning arm
.
Python configure
./configure --prefix=/data/python/python-3.8.11 --host=arm-linux-gnueabihf --build=x86_64-linux-gnu --without-ensurepip ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no CFLAGS="-march=armv7-a" LDFLAGS="-march=armv7-a"
/data/python/python-native-3.7.11/bin/python3 -m crossenv --env CFLAGS="-I/data/python/arm_tools/libffi/include" --env LDFLAGS="-L/data/python/arm_tools/libffi/lib" /data/python/python-3.7.11/bin/python3 cross_venv_3.7.11
>>> import platform
>>> platform.uname()
uname_result(system='Linux', node='build', release='', version='', machine='arm', processor='arm')
Native armv7l returns this
>>> import platform
>>> platform.uname()
uname_result(system='Linux', node='rasp-damien', release='5.10.17-v7+', version='#1421 SMP Thu May 27 13:59:01 BST 2021', machine='armv7l', processor='')
Collecting cryptography==3.4.4
Downloading cryptography-3.4.4.tar.gz (545 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
�[91m ERROR: Command errored out with exit status 1:
command: /root/tools_build/pythonCross/cross_venv/cross/bin/python3.8 /root/tools_build/pythonCross/cross_venv/build/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hzx8jj8x/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4'
cwd: None
Complete output (80 lines):
Collecting setuptools>=40.6.0
Using cached setuptools-54.1.2-py3-none-any.whl (785 kB)
Collecting wheel
Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Collecting cffi>=1.12
Downloading cffi-1.14.5.tar.gz (475 kB)
Collecting setuptools-rust>=0.11.4
Downloading setuptools_rust-0.12.1-py3-none-any.whl (22 kB)
Collecting pycparser
Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting semantic-version>=2.6.0
Downloading semantic_version-2.8.5-py2.py3-none-any.whl (15 kB)
Collecting toml>=0.9.0
Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Using legacy 'setup.py install' for cffi, since package 'wheel' is not installed.
Installing collected packages: setuptools, wheel, pycparser, cffi, semantic-version, toml, setuptools-rust
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /root/tools_build/pythonCross/cross_venv/cross/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1zd6kvzn/cffi/setup.py'"'"'; file='"'"'/tmp/pip-install-1zd6kvzn/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-mzn529bp/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-hzx8jj8x/overlay --compile --install-headers /tmp/pip-build-env-hzx8jj8x/overlay/include/site/python3.8/cffi
cwd: /tmp/pip-install-1zd6kvzn/cffi/
Complete output (54 lines):
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing libffi.pc' to the PKG_CONFIG_PATH environment variable No package 'libffi' found Package libffi was not found in the pkg-config search path. Perhaps you should add the directory containing
libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing libffi.pc' to the PKG_CONFIG_PATH environment variable No package 'libffi' found Package libffi was not found in the pkg-config search path. Perhaps you should add the directory containing
libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
running install
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.8
creating build/lib.linux-aarch64-3.8/cffi
copying cffi/init.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/api.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/commontypes.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/cparser.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/error.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/ffiplatform.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/lock.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/model.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/pkgconfig.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/recompiler.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/vengine_gen.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/verifier.py -> build/lib.linux-aarch64-3.8/cffi
copying cffi/_cffi_include.h -> build/lib.linux-aarch64-3.8/cffi
copying cffi/parse_c_type.h -> build/lib.linux-aarch64-3.8/cffi
copying cffi/_embedding.h -> build/lib.linux-aarch64-3.8/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-aarch64-3.8/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-aarch64-3.8
creating build/temp.linux-aarch64-3.8/c
aarch64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/tools_build/pythonCross/cross_venv/cross/include -I/usr/aarch64-linux-gnu/python3.8/include/python3.8 -c c/_cffi_backend.c -o build/temp.linux-aarch64-3.8/c/_cffi_backend.o
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.
error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /root/tools_build/pythonCross/cross_venv/cross/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1zd6kvzn/cffi/setup.py'"'"'; file='"'"'/tmp/pip-install-1zd6kvzn/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-mzn529bp/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-hzx8jj8x/overlay --compile --install-headers /tmp/pip-build-env-hzx8jj8x/overlay/include/site/python3.8/cffi Check the logs for full command output.
If I set a PKG_CONFIG_PATH to find libffi.pc, which 1.0 finds just fine, then I get further erros:
Collecting cryptography==3.4.4
Using cached cryptography-3.4.4.tar.gz (545 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1:
command: /root/tools_build/pythonCross/cross_env/cross/bin/python3.8 /root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmphaxon7ft
cwd: /tmp/pip-install-naq2drvu/cryptography
Complete output (56 lines):
=============================DEBUG ASSISTANCE=============================
If you are seeing a compilation error please try the following steps to
successfully install cryptography:
1) Upgrade to the latest pip and try again. This will fix errors for most
users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
2) Read https://cryptography.io/en/latest/installation.html for specific
instructions for your platform.
3) Check our frequently asked questions for more information:
https://cryptography.io/en/latest/faq.html
4) Ensure you have a recent Rust toolchain installed:
https://cryptography.io/en/latest/installation.html#rust
5) If you are experiencing issues with Rust for *this release only* you may
set the environment variable `CRYPTOGRAPHY_DONT_BUILD_RUST=1`.
=============================DEBUG ASSISTANCE=============================
Traceback (most recent call last):
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
main()
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 157, in prepare_metadata_for_build_wheel
self.run_setup()
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 142, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 71, in <module>
setup(
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/setuptools/dist.py", line 429, in __init__
_Distribution.__init__(self, {
File "/usr/lib/python3.8/distutils/dist.py", line 292, in __init__
self.finalize_options()
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/setuptools/dist.py", line 721, in finalize_options
ep(self)
File "/root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/setuptools/dist.py", line 728, in _finalize_setup_keywords
ep.load()(self, ep.name, value)
File "/tmp/pip-build-env-c8q5738l/overlay/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 219, in cffi_modules
add_cffi_module(dist, cffi_module)
File "/tmp/pip-build-env-c8q5738l/overlay/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
execfile(build_file_name, mod_vars)
File "/tmp/pip-build-env-c8q5738l/overlay/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 25, in execfile
exec(code, glob, glob)
File "src/_cffi_src/build_openssl.py", line 76, in <module>
ffi = build_ffi_for_binding(
File "src/_cffi_src/utils.py", line 53, in build_ffi_for_binding
ffi = build_ffi(
File "src/_cffi_src/utils.py", line 73, in build_ffi
ffi = FFI()
File "/tmp/pip-build-env-c8q5738l/overlay/lib/python3.8/site-packages/cffi/api.py", line 48, in __init__
import _cffi_backend as backend
ModuleNotFoundError: No module named '_cffi_backend'
----------------------------------------
ERROR: Command errored out with exit status 1: /root/tools_build/pythonCross/cross_env/cross/bin/python3.8 /root/tools_build/pythonCross/cross_env/build/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmphaxon7ft Check the logs for full command output.
Hello,
Building wheel for bcrypt did not run successfully.
build-pip install bcrypt
Collecting bcrypt
Using cached bcrypt-3.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (61 kB)
Collecting six>=1.4.1
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Requirement already satisfied: cffi>=1.1 in ./venv/build/lib/python3.8/site-packages (from bcrypt) (1.15.0)
Requirement already satisfied: pycparser in ./venv/build/lib/python3.8/site-packages (from cffi>=1.1->bcrypt) (2.21)
Installing collected packages: six, bcrypt
Successfully installed bcrypt-3.2.0 six-1.16.0
pip -v install bcrypt
Using pip 22.0.2 from /home/sylvain/venv/build/lib/python3.8/site-packages/pip (python 3.8)
Collecting bcrypt
Using cached bcrypt-3.2.0.tar.gz (42 kB)
Running command pip subprocess to install build dependencies
Collecting setuptools>=40.8.0
Using cached setuptools-60.6.0-py3-none-any.whl (953 kB)
Collecting wheel
Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting cffi>=1.1
Using cached cffi-1.15.0-cp38-cp38-linux_aarch64.whl
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: wheel, setuptools, pycparser, cffi
Successfully installed cffi-1.15.0 pycparser-2.21 setuptools-60.6.0 wheel-0.37.1
Installing build dependencies ... done
Running command Getting requirements to build wheel
Getting requirements to build wheel ... done
Running command Preparing metadata (pyproject.toml)
running dist_info
creating /tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info
writing /tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info/dependency_links.txt
writing requirements to /tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info/requires.txt
writing top-level names to /tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info/top_level.txt
writing manifest file '/tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'requirements.txt'
warning: no previously-included files found matching 'release.py'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching 'mypy.ini'
no previously-included directories found matching '.travis'
writing manifest file '/tmp/pip-modern-metadata-c_tiv_t7/bcrypt.egg-info/SOURCES.txt'
creating '/tmp/pip-modern-metadata-c_tiv_t7/bcrypt.dist-info'
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cffi>=1.1 in ./venv/cross/lib/python3.8/site-packages (from bcrypt) (1.15.0)
Collecting six>=1.4.1
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Requirement already satisfied: pycparser in ./venv/cross/lib/python3.8/site-packages (from cffi>=1.1->bcrypt) (2.21)
Building wheels for collected packages: bcrypt
Running command Building wheel for bcrypt (pyproject.toml)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.8
creating build/lib.linux-aarch64-3.8/bcrypt
copying src/bcrypt/init.py -> build/lib.linux-aarch64-3.8/bcrypt
copying src/bcrypt/about.py -> build/lib.linux-aarch64-3.8/bcrypt
copying src/bcrypt/py.typed -> build/lib.linux-aarch64-3.8/bcrypt
Traceback (most recent call last):
File "/home/sylvain/venv/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in
main()
File "/home/sylvain/venv/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/sylvain/venv/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/home/sylvain/venv/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 212, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/home/sylvain/venv/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 198, in _build_with_temp_dir
self.run_setup()
File "/home/sylvain/venv/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 143, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 46, in
setup(
File "/home/sylvain/venv/build/lib/python3.8/site-packages/setuptools/init.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/local/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/sylvain/venv/build/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/sylvain/venv/build/lib/python3.8/site-packages/setuptools/command/build_py.py", line 58, in run
self.build_package_data()
File "/home/sylvain/venv/build/lib/python3.8/site-packages/setuptools/command/build_py.py", line 132, in build_package_data
srcfile in self.distribution.convert_2to3_doctests):
AttributeError: 'Distribution' object has no attribute 'convert_2to3_doctests'
error: subprocess-exited-with-error× Building wheel for bcrypt (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/sylvain/venv/cross/bin/python3 /home/sylvain/venv/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpitmrotvb
cwd: /tmp/pip-install-t6100zow/bcrypt_6504c975743f46aea1af08b10f417bb0
Building wheel for bcrypt (pyproject.toml) ... error
ERROR: Failed building wheel for bcrypt
Failed to build bcrypt
ERROR: Could not build wheels for bcrypt, which is required to install pyproject.toml-based projects
How can I fix that?
Regards
I can't install bcrypt (and any other libraries, that used cffi in compile time).
Requirement already satisfied: cffi>=1.1 in ./venv/cross/lib/python3.6/site-packages (from bcrypt) Requirement already satisfied: six>=1.4.1 in ./venv/cross/lib/python3.6/site-packages (from bcrypt) Requirement already satisfied: pycparser in ./venv/cross/lib/python3.6/site-packages (from cffi>=1.1->bcrypt) Installing collected packages: bcrypt Running setup.py install for bcrypt ... error Complete output from command /root/source/venv/cross/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-y3sc7gpk/bcrypt/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-pw2_2nbg-record/install-record.txt --single-version-externally-managed --compile --install-headers /root/source/venv/cross/include/site/python3.6/bcrypt: ******************************************************* * Crossenv has leaked into another Python interpreter! * You should probably file a bug report. * Version 3.6.0 (default, Nov 7 2018, 18:40:18) [GCC 5.4.0 20160609] * Executable /root/source/venv/cross/bin/python3 ******************************************************* Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-build-y3sc7gpk/bcrypt/setup.py", line 238, in <module> **keywords_with_side_effects(sys.argv) File "/root/source/staging_dir/target-mipsel_24kc_musl-1.1.16/host/lib/python3.6/distutils/core.py", line 108, in setup _setup_distribution = dist = klass(attrs) File "/root/source/venv/build/lib/python3.6/site-packages/setuptools/dist.py", line 318, in __init__ _Distribution.__init__(self, attrs) File "/root/source/staging_dir/target-mipsel_24kc_musl-1.1.16/host/lib/python3.6/distutils/dist.py", line 281, in __init__ self.finalize_options() File "/root/source/venv/build/lib/python3.6/site-packages/setuptools/dist.py", line 376, in finalize_options ep.load()(self, ep.name, value) File "/root/source/venv/cross/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 204, in cffi_modules add_cffi_module(dist, cffi_module) File "/root/source/venv/cross/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module execfile(build_file_name, mod_vars) File "/root/source/venv/cross/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 25, in execfile exec(code, glob, glob) File "src/build_bcrypt.py", line 21, in <module> ffi = FFI() File "/root/source/venv/cross/lib/python3.6/site-packages/cffi/api.py", line 46, in __init__ import _cffi_backend as backend ImportError: /root/source/venv/cross/lib/python3.6/site-packages/_cffi_backend.cpython-36.so: wrong ELF class: ELFCLASS32
As you can see, installing via pip completes for cffi, and then fails for bcrypt because _cffi_backend.cpython-36.so
compiled for HOST platform, but used in compile time on the BUILD machine.
There don't appear to be major changes, but some small fixes are needed.
hi, crossenv team:
I want to cross compile numpy-1.19.4 in mips platform, and I have the issue:
Running from numpy source directory.
Note: if you need reliable uninstall behavior, then install
with pip instead of using setup.py install
:
pip install .
(from a git repo or downloaded sourcepip install numpy
(last NumPy release on PyPi)Cythonizing sources
numpy/random/_bounded_integers.pxd.in has not changed
numpy/random/bit_generator.pyx has not changed
numpy/random/_common.pyx has not changed
numpy/random/_mt19937.pyx has not changed
numpy/random/mtrand.pyx has not changed
numpy/random/_pcg64.pyx has not changed
numpy/random/_philox.pyx has not changed
numpy/random/_sfc64.pyx has not changed
numpy/random/_bounded_integers.pyx.in has not changed
numpy/random/_bounded_integers.pyx has not changed
numpy/random/_generator.pyx has not changed
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
C compiler: ccache mipsel-linux-uclibc-gcc -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -Wno-unused-result -Wsign-compare -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -I/home/lwd/py3/src/router/newopenssl/include -I/home/lwd/py3/src/router/zlib -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -I/home/lwd/py3/src/router/newopenssl/include -I/home/lwd/py3/src/router/zlib -I/home/lwd/py3/src/router/mipsel-uclibc/install/libffi/usr/lib/libffi-3.2.1/include -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -fPIC
creating /tmp/tmp1r9ipjim/tmp
creating /tmp/tmp1r9ipjim/tmp/tmp1r9ipjim
compile options: '-MMD -MF /tmp/tmp1r9ipjim/file.c.d -c'
ccache: /tmp/tmp1r9ipjim/file.c
libraries mkl_rt not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
blis_info:
libraries blis not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
openblas_info:
libraries openblas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
libraries tatlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
atlas_3_10_blas_info:
libraries satlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
accelerate_info:
NOT AVAILABLE
/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1914: UserWarning:
Optimized (vendor) Blas libraries are not found.
Falls back to netlib Blas library which has worse performance.
A better performance should be easily gained by switching
Blas library.
if self._calc_info(blas):
blas_info:
libraries blas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1914: UserWarning:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
if self._calc_info(blas):
blas_src_info:
NOT AVAILABLE
/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1914: UserWarning:
Blas (http://www.netlib.org/blas/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [blas_src]) or by setting
the BLAS_SRC environment variable.
if self._calc_info(blas):
NOT AVAILABLE
non-existing path in 'numpy/distutils': 'site.cfg'
lapack_opt_info:
lapack_mkl_info:
libraries mkl_rt not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
openblas_lapack_info:
libraries openblas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
openblas_clapack_info:
libraries openblas,lapack not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
flame_info:
libraries flame not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries tatlas,tatlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries lapack_atlas not found in /usr/local/lib
libraries tatlas,tatlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib/sse2
libraries tatlas,tatlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib
libraries tatlas,tatlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2
libraries tatlas,tatlas not found in /usr/lib/i386-linux-gnu/sse2
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu
libraries tatlas,tatlas not found in /usr/lib/i386-linux-gnu
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
NOT AVAILABLE
atlas_3_10_info:
libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries satlas,satlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries lapack_atlas not found in /usr/local/lib
libraries satlas,satlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib/sse2
libraries satlas,satlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib
libraries satlas,satlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2
libraries satlas,satlas not found in /usr/lib/i386-linux-gnu/sse2
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu
libraries satlas,satlas not found in /usr/lib/i386-linux-gnu
<class 'numpy.distutils.system_info.atlas_3_10_info'>
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries ptf77blas,ptcblas,atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries lapack_atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/i386-linux-gnu/sse2
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/i386-linux-gnu
<class 'numpy.distutils.system_info.atlas_threads_info'>
NOT AVAILABLE
atlas_info:
libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries f77blas,cblas,atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib
libraries lapack_atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2
libraries f77blas,cblas,atlas not found in /usr/lib/i386-linux-gnu/sse2
libraries lapack_atlas not found in /usr/lib/i386-linux-gnu
libraries f77blas,cblas,atlas not found in /usr/lib/i386-linux-gnu
<class 'numpy.distutils.system_info.atlas_info'>
NOT AVAILABLE
lapack_info:
libraries lapack not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu']
NOT AVAILABLE
/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1748: UserWarning:
Lapack (http://www.netlib.org/lapack/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [lapack]) or by setting
the LAPACK environment variable.
return getattr(self, 'calc_info{}'.format(name))()
lapack_src_info:
NOT AVAILABLE
/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1748: UserWarning:
Lapack (http://www.netlib.org/lapack/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [lapack_src]) or by setting
the LAPACK_SRC environment variable.
return getattr(self, 'calc_info{}'.format(name))()
NOT AVAILABLE
numpy_linalg_lapack_lite:
FOUND:
language = c
/usr/local/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
warnings.warn(msg)
running install
Checking .pth file support in /home/lwd/python_lib/tmp/lib/python3.8/site-packages
/home/lwd/python_lib/test/cross_venv/cross/bin/python3.8 -E -c pass
TEST FAILED: /home/lwd/python_lib/tmp/lib/python3.8/site-packages does NOT support .pth files
bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/home/lwd/python_lib/tmp/lib/python3.8/site-packages
and your PYTHONPATH environment variable currently contains:
'/home/lwd/python_lib/test/cross_venv/lib:/usr/local/lib/python3.8'
Here are some of your options for correcting the problem:
You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://setuptools.readthedocs.io/en/latest/easy_install.html#custom-installation-locations
Please make the appropriate changes for your system and try again.
running bdist_egg
running egg_info
running build_src
build_src
building py_modules sources
building library "npymath" sources
Could not locate executable gfortran
Could not locate executable f95
Could not locate executable ifort
Could not locate executable ifc
Could not locate executable lf95
Could not locate executable pgfortran
Could not locate executable nvfortran
Could not locate executable f90
Could not locate executable f77
Could not locate executable fort
Could not locate executable efort
Could not locate executable efc
Could not locate executable g77
Could not locate executable g95
Could not locate executable pathf95
Could not locate executable nagfor
don't know how to compile Fortran code on platform 'posix'
adding 'build/src.linux-mipsel-3.8/numpy/core/src/npymath' to include_dirs.
None - nothing done with h_files = ['build/src.linux-mipsel-3.8/numpy/core/src/npymath/npy_math_internal.h']
building library "npysort" sources
adding 'build/src.linux-mipsel-3.8/numpy/core/src/common' to include_dirs.
None - nothing done with h_files = ['build/src.linux-mipsel-3.8/numpy/core/src/common/npy_sort.h', 'build/src.linux-mipsel-3.8/numpy/core/src/common/npy_partition.h', 'build/src.linux-mipsel-3.8/numpy/core/src/common/npy_binsearch.h']
building library "npyrandom" sources
building extension "numpy.core._multiarray_tests" sources
building extension "numpy.core._multiarray_umath" sources
Traceback (most recent call last):
File "setup.py", line 508, in
setup_package()
File "setup.py", line 500, in setup_package
setup(**metadata)
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/core.py", line 169, in setup
return old_setup(**new_attr)
File "/home/lwd/python_lib/test/cross_venv/build/lib/python3.8/site-packages/setuptools/init.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/local/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/install.py", line 60, in run
r = self.setuptools_run()
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/install.py", line 54, in setuptools_run
self.do_egg_install()
File "/home/lwd/python_lib/test/cross_venv/build/lib/python3.8/site-packages/setuptools/command/install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/lwd/python_lib/test/cross_venv/build/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 164, in run
self.run_command("egg_info")
File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/egg_info.py", line 24, in run
self.run_command("build_src")
File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 144, in run
self.build_sources()
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 161, in build_sources
self.build_extension_sources(ext)
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 318, in build_extension_sources
sources = self.generate_sources(sources, ext)
File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 378, in generate_sources
source = func(extension, build_dir)
File "numpy/core/setup.py", line 436, in generate_config_h
moredefs, ignored = cocache.check_types(config_cmd, ext, build_dir)
File "numpy/core/setup.py", line 47, in check_types
out = check_types(*a, **kw)
File "numpy/core/setup.py", line 292, in check_types
raise SystemError(
SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.
I try to apt install python3-dev and pypy,but it does not works. can you help me for this issue? thank you.
I'm trying to cross compile Numpy with OpenBLAS. I've already cross compiles OpenBLAS and installed it into python_armhf/
, i.e. headers are in python_armhf/include
and binaries are in python_armhf/lib
. I create the crossenv with
path/to/build-python -m crossenv \
--env=CPATH:=python_armhf/include \
--env=LIBRARY_PATH:=python_armhf/lib \
path/to/host-python env
However, when I'm in the crossenv neither of those environment variables are set, and env/cross/lib
does not contain anything from python_armhf/lib
. Consequently, the Numpy build fails due to missing OpenBLAS.
Am I misunderstanding how --env is supposed to work? How can I specify where to find cross compiled libraries in the crossenv?
I'm trying install jupyter in cross-python, failed, I will find other ways to get it.
But there's a warning during installation which maybe helpful to improve crossenv.
(cross) ➜ ~/projects/crossVenv4Py pip install jupyter
Looking in indexes: http://mirrors.aliyun.com/pypi/simple/
Collecting jupyter
Downloading http://mirrors.aliyun.com/pypi/packages/83/df/0f5dd132200728a86190397e1ea87cd76244e42d39ec5e88efd25b2abd7e/jupyter-1.0.0-py2.py3-none-any.whl
Collecting ipywidgets (from jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/30/9a/a008c7b1183fac9e52066d80a379b3c64eab535bd9d86cdc29a0b766fd82/ipywidgets-7.4.2-py2.py3-none-any.whl (111kB)
100% |████████████████████████████████| 112kB 1.5MB/s
Collecting notebook (from jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/a2/5d/d1907cd32ac00b5ead56f6e61d9794fa60ef105a22ac5da6e7556011580f/notebook-5.7.2-py2.py3-none-any.whl (9.0MB)
100% |████████████████████████████████| 9.0MB 5.1MB/s
Collecting jupyter-console (from jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/cb/ee/6374ae8c21b7d0847f9c3722dcdfac986b8e54fa9ad9ea66e1eb6320d2b8/jupyter_console-6.0.0-py2.py3-none-any.whl
Collecting qtconsole (from jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/e0/7a/8aefbc0ed078dec7951ac9a06dcd1869243ecd7bcbce26fa47bf5e469a8f/qtconsole-4.4.3-py2.py3-none-any.whl (113kB)
100% |████████████████████████████████| 122kB 5.5MB/s
Collecting ipykernel (from jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/d8/b0/f0be5c5ab335196f5cce96e5b889a4fcf5bfe462eb0acc05cd7e2caf65eb/ipykernel-5.1.0-py3-none-any.whl (113kB)
100% |████████████████████████████████| 122kB 6.3MB/s
Requirement already satisfied: nbconvert in ./cross/lib/python3.7/site-packages (from jupyter) (5.4.0)
Collecting widgetsnbextension~=3.4.0 (from ipywidgets->jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/8a/81/35789a3952afb48238289171728072d26d6e76649ddc8b3588657a2d78c1/widgetsnbextension-3.4.2-py2.py3-none-any.whl (2.2MB)
100% |████████████████████████████████| 2.2MB 5.2MB/s
Requirement already satisfied: nbformat>=4.2.0 in ./cross/lib/python3.7/site-packages (from ipywidgets->jupyter) (4.4.0)
Requirement already satisfied: traitlets>=4.3.1 in ./cross/lib/python3.7/site-packages (from ipywidgets->jupyter) (4.3.2)
Requirement already satisfied: ipython>=4.0.0; python_version >= "3.3" in ./cross/lib/python3.7/site-packages (from ipywidgets->jupyter) (7.1.1)
Collecting pyzmq>=17 (from notebook->jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/b9/6a/bc9277b78f5c3236e36b8c16f4d2701a7fd4fa2eb697159d3e0a3a991573/pyzmq-17.1.2.tar.gz (1.1MB)
100% |████████████████████████████████| 1.1MB 5.2MB/s
Requirement already satisfied: tornado>=4 in ./cross/lib/python3.7/site-packages (from notebook->jupyter) (5.1.1)
Requirement already satisfied: jinja2 in ./cross/lib/python3.7/site-packages (from notebook->jupyter) (2.10)
Requirement already satisfied: terminado>=0.8.1 in ./cross/lib/python3.7/site-packages (from notebook->jupyter) (0.8.1)
Collecting jupyter-client>=5.2.0 (from notebook->jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/94/dd/fe6c4d683b09eb05342bd2816b7779663f71762b4fa9c2d5203d35d17354/jupyter_client-5.2.3-py2.py3-none-any.whl (89kB)
100% |████████████████████████████████| 92kB 7.5MB/s
Collecting prometheus-client (from notebook->jupyter)
Downloading http://mirrors.aliyun.com/pypi/packages/61/84/9aa657b215b04f21a72ca8e50ff159eef9795096683e4581a357baf4dde6/prometheus_client-0.4.2.tar.gz
Requirement already satisfied: Send2Trash in ./cross/lib/python3.7/site-packages (from notebook->jupyter) (1.5.0)
Requirement already satisfied: ipython-genutils in ./cross/lib/python3.7/site-packages (from notebook->jupyter) (0.2.0)
Requirement already satisfied: jupyter-core>=4.4.0 in ./cross/lib/python3.7/site-packages (from notebook->jupyter) (4.4.0)
Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in ./cross/lib/python3.7/site-packages (from jupyter-console->jupyter) (2.0.7)
Requirement already satisfied: pygments in ./cross/lib/python3.7/site-packages (from jupyter-console->jupyter) (2.2.0)
Requirement already satisfied: bleach in ./cross/lib/python3.7/site-packages (from nbconvert->jupyter) (3.0.2)
Requirement already satisfied: pandocfilters>=1.4.1 in ./cross/lib/python3.7/site-packages (from nbconvert->jupyter) (1.4.2)
Requirement already satisfied: defusedxml in ./cross/lib/python3.7/site-packages (from nbconvert->jupyter) (0.5.0)
Requirement already satisfied: testpath in ./cross/lib/python3.7/site-packages (from nbconvert->jupyter) (0.4.2)
Requirement already satisfied: entrypoints>=0.2.2 in ./cross/lib/python3.7/site-packages (from nbconvert->jupyter) (0.2.3)
Requirement already satisfied: mistune>=0.8.1 in ./cross/lib/python3.7/site-packages (from nbconvert->jupyter) (0.8.4)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in ./cross/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets->jupyter) (2.6.0)
Requirement already satisfied: decorator in ./cross/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets->jupyter) (4.3.0)
Requirement already satisfied: six in ./cross/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets->jupyter) (1.11.0)
Requirement already satisfied: setuptools>=18.5 in ./cross/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets->jupyter) (39.0.1)
Requirement already satisfied: backcall in ./cross/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets->jupyter) (0.1.0)
Requirement already satisfied: jedi>=0.10 in ./cross/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets->jupyter) (0.13.1)
Requirement already satisfied: pickleshare in ./cross/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets->jupyter) (0.7.5)
Requirement already satisfied: pexpect; sys_platform != "win32" in ./cross/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets->jupyter) (4.6.0)
Requirement already satisfied: MarkupSafe>=0.23 in ./cross/lib/python3.7/site-packages (from jinja2->notebook->jupyter) (1.1.0)
Requirement already satisfied: ptyprocess; os_name != "nt" in ./cross/lib/python3.7/site-packages (from terminado>=0.8.1->notebook->jupyter) (0.6.0)
Requirement already satisfied: python-dateutil>=2.1 in ./cross/lib/python3.7/site-packages (from jupyter-client>=5.2.0->notebook->jupyter) (2.7.5)
Requirement already satisfied: wcwidth in ./cross/lib/python3.7/site-packages (from prompt-toolkit<2.1.0,>=2.0.0->jupyter-console->jupyter) (0.1.7)
Requirement already satisfied: webencodings in ./cross/lib/python3.7/site-packages (from bleach->nbconvert->jupyter) (0.5.1)
Requirement already satisfied: parso>=0.3.0 in ./cross/lib/python3.7/site-packages (from jedi>=0.10->ipython>=4.0.0; python_version >= "3.3"->ipywidgets->jupyter) (0.3.1)
Installing collected packages: pyzmq, jupyter-client, ipykernel, prometheus-client, notebook, widgetsnbextension, ipywidgets, jupyter-console, qtconsole, jupyter
Running setup.py install for pyzmq ... error
Complete output from command /home/xys/projects/crossVenv4Py/cross/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-5p6a3wit/pyzmq/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-g19xjnhf/install-record.txt --single-version-externally-managed --compile --install-headers /home/xys/projects/crossVenv4Py/cross/include/site/python3.7/pyzmq:
*******************************************************
* Crossenv has leaked into another Python interpreter!
* You should probably file a bug report.
* Version 3.7.1 (default, Nov 20 2018, 20:36:43)
[GCC 5.4.0 20160609]
* Executable /home/xys/projects/crossVenv4Py/cross/bin/python3.7
*******************************************************
running install
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.7
creating build/lib.linux-aarch64-3.7/zmq
copying zmq/__init__.py -> build/lib.linux-aarch64-3.7/zmq
copying zmq/error.py -> build/lib.linux-aarch64-3.7/zmq
copying zmq/_future.py -> build/lib.linux-aarch64-3.7/zmq
copying zmq/decorators.py -> build/lib.linux-aarch64-3.7/zmq
creating build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/frame.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/stopwatch.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/__init__.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/poll.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/socket.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/version.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/attrsettr.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/tracker.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/context.py -> build/lib.linux-aarch64-3.7/zmq/sugar
copying zmq/sugar/constants.py -> build/lib.linux-aarch64-3.7/zmq/sugar
creating build/lib.linux-aarch64-3.7/zmq/eventloop
copying zmq/eventloop/zmqstream.py -> build/lib.linux-aarch64-3.7/zmq/eventloop
copying zmq/eventloop/_deprecated.py -> build/lib.linux-aarch64-3.7/zmq/eventloop
copying zmq/eventloop/__init__.py -> build/lib.linux-aarch64-3.7/zmq/eventloop
copying zmq/eventloop/ioloop.py -> build/lib.linux-aarch64-3.7/zmq/eventloop
copying zmq/eventloop/future.py -> build/lib.linux-aarch64-3.7/zmq/eventloop
creating build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado
copying zmq/eventloop/minitornado/util.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado
copying zmq/eventloop/minitornado/concurrent.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado
copying zmq/eventloop/minitornado/__init__.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado
copying zmq/eventloop/minitornado/ioloop.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado
copying zmq/eventloop/minitornado/log.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado
copying zmq/eventloop/minitornado/stack_context.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado
creating build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado/platform
copying zmq/eventloop/minitornado/platform/posix.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado/platform
copying zmq/eventloop/minitornado/platform/__init__.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado/platform
copying zmq/eventloop/minitornado/platform/auto.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado/platform
copying zmq/eventloop/minitornado/platform/interface.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado/platform
copying zmq/eventloop/minitornado/platform/common.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado/platform
copying zmq/eventloop/minitornado/platform/windows.py -> build/lib.linux-aarch64-3.7/zmq/eventloop/minitornado/platform
creating build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/win32.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/z85.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/__init__.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/jsonapi.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/constant_names.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/garbage.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/sixcerpt.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/strtypes.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/interop.py -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/monitor.py -> build/lib.linux-aarch64-3.7/zmq/utils
creating build/lib.linux-aarch64-3.7/zmq/log
copying zmq/log/__init__.py -> build/lib.linux-aarch64-3.7/zmq/log
copying zmq/log/handlers.py -> build/lib.linux-aarch64-3.7/zmq/log
creating build/lib.linux-aarch64-3.7/zmq/green
copying zmq/green/__init__.py -> build/lib.linux-aarch64-3.7/zmq/green
copying zmq/green/poll.py -> build/lib.linux-aarch64-3.7/zmq/green
copying zmq/green/device.py -> build/lib.linux-aarch64-3.7/zmq/green
copying zmq/green/core.py -> build/lib.linux-aarch64-3.7/zmq/green
creating build/lib.linux-aarch64-3.7/zmq/green/eventloop
copying zmq/green/eventloop/zmqstream.py -> build/lib.linux-aarch64-3.7/zmq/green/eventloop
copying zmq/green/eventloop/__init__.py -> build/lib.linux-aarch64-3.7/zmq/green/eventloop
copying zmq/green/eventloop/ioloop.py -> build/lib.linux-aarch64-3.7/zmq/green/eventloop
creating build/lib.linux-aarch64-3.7/zmq/auth
copying zmq/auth/thread.py -> build/lib.linux-aarch64-3.7/zmq/auth
copying zmq/auth/base.py -> build/lib.linux-aarch64-3.7/zmq/auth
copying zmq/auth/__init__.py -> build/lib.linux-aarch64-3.7/zmq/auth
copying zmq/auth/ioloop.py -> build/lib.linux-aarch64-3.7/zmq/auth
copying zmq/auth/certs.py -> build/lib.linux-aarch64-3.7/zmq/auth
creating build/lib.linux-aarch64-3.7/zmq/auth/asyncio
copying zmq/auth/asyncio/__init__.py -> build/lib.linux-aarch64-3.7/zmq/auth/asyncio
creating build/lib.linux-aarch64-3.7/zmq/asyncio
copying zmq/asyncio/__init__.py -> build/lib.linux-aarch64-3.7/zmq/asyncio
creating build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_cffi_backend.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_security.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_win32_shim.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_zmqstream.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_includes.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_pubsub.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/__init__.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_ssh.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_z85.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_auth.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_context.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_constants.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_imports.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_reqrep.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_socket.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_version.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_future.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_monqueue.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_ioloop.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_pair.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_multipart.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_message.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_etc.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_poll.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_monitor.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_log.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_error.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_device.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_retry_eintr.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_decorators.py -> build/lib.linux-aarch64-3.7/zmq/tests
copying zmq/tests/test_draft.py -> build/lib.linux-aarch64-3.7/zmq/tests
creating build/lib.linux-aarch64-3.7/zmq/tests/asyncio
copying zmq/tests/asyncio/__init__.py -> build/lib.linux-aarch64-3.7/zmq/tests/asyncio
copying zmq/tests/asyncio/test_asyncio.py -> build/lib.linux-aarch64-3.7/zmq/tests/asyncio
copying zmq/tests/asyncio/_test_asyncio.py -> build/lib.linux-aarch64-3.7/zmq/tests/asyncio
creating build/lib.linux-aarch64-3.7/zmq/devices
copying zmq/devices/monitoredqueuedevice.py -> build/lib.linux-aarch64-3.7/zmq/devices
copying zmq/devices/proxydevice.py -> build/lib.linux-aarch64-3.7/zmq/devices
copying zmq/devices/monitoredqueue.py -> build/lib.linux-aarch64-3.7/zmq/devices
copying zmq/devices/__init__.py -> build/lib.linux-aarch64-3.7/zmq/devices
copying zmq/devices/basedevice.py -> build/lib.linux-aarch64-3.7/zmq/devices
creating build/lib.linux-aarch64-3.7/zmq/backend
copying zmq/backend/__init__.py -> build/lib.linux-aarch64-3.7/zmq/backend
copying zmq/backend/select.py -> build/lib.linux-aarch64-3.7/zmq/backend
creating build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/_cffi.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/__init__.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/error.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/utils.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/devices.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/socket.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/message.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/_poll.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/context.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/constants.py -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
creating build/lib.linux-aarch64-3.7/zmq/backend/cython
copying zmq/backend/cython/__init__.py -> build/lib.linux-aarch64-3.7/zmq/backend/cython
creating build/lib.linux-aarch64-3.7/zmq/ssh
copying zmq/ssh/__init__.py -> build/lib.linux-aarch64-3.7/zmq/ssh
copying zmq/ssh/tunnel.py -> build/lib.linux-aarch64-3.7/zmq/ssh
copying zmq/ssh/forward.py -> build/lib.linux-aarch64-3.7/zmq/ssh
copying zmq/utils/buffers.pxd -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/mutex.h -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/ipcmaxlen.h -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/zmq_compat.h -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/getpid_compat.h -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/pyversion_compat.h -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/utils/zmq_constants.h -> build/lib.linux-aarch64-3.7/zmq/utils
copying zmq/devices/monitoredqueue.pxd -> build/lib.linux-aarch64-3.7/zmq/devices
copying zmq/backend/cffi/_cdefs.h -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cffi/_verify.c -> build/lib.linux-aarch64-3.7/zmq/backend/cffi
copying zmq/backend/cython/message.pxd -> build/lib.linux-aarch64-3.7/zmq/backend/cython
copying zmq/backend/cython/checkrc.pxd -> build/lib.linux-aarch64-3.7/zmq/backend/cython
copying zmq/backend/cython/context.pxd -> build/lib.linux-aarch64-3.7/zmq/backend/cython
copying zmq/backend/cython/libzmq.pxd -> build/lib.linux-aarch64-3.7/zmq/backend/cython
copying zmq/backend/cython/socket.pxd -> build/lib.linux-aarch64-3.7/zmq/backend/cython
copying zmq/backend/cython/constant_enums.pxi -> build/lib.linux-aarch64-3.7/zmq/backend/cython
copying zmq/backend/cython/constants.pxi -> build/lib.linux-aarch64-3.7/zmq/backend/cython
running build_ext
running configure
Did not find libzmq via pkg-config.
{'libraries': ['zmq'], 'include_dirs': ['/home/xys/projects/crossVenv4Py/cross/include'], 'library_dirs': ['/home/xys/projects/crossVenv4Py/cross/lib'], 'runtime_library_dirs': ['/home/xys/projects/crossVenv4Py/cross/lib'], 'extra_link_args': []}
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -I/home/xys/projects/crossVenv4Py/cross/include -c build/temp.linux-aarch64-3.7/scratch/check_sys_un.c -o build/temp.linux-aarch64-3.7/scratch/check_sys_un.o
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE build/temp.linux-aarch64-3.7/scratch/check_sys_un.o -L/home/xys/projects/crossVenv4Py/cross/lib -R/home/xys/projects/crossVenv4Py/cross/lib -o build/temp.linux-aarch64-3.7/scratch/check_sys_un
aarch64-linux-android-gcc: error: unrecognized command line option '-R'
Warning: No sys/un.h, IPC_PATH_MAX_LEN will be undefined: command 'aarch64-linux-android-gcc' failed with exit status 1
************************************************
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir:
creating build/temp.linux-aarch64-3.7/scratch/tmp
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -c /tmp/timer_createft0sob19.c -o build/temp.linux-aarch64-3.7/scratch/tmp/timer_createft0sob19.o
/tmp/timer_createft0sob19.c:1:1: warning: return type defaults to 'int' [-Wreturn-type]
main (int argc, char **argv) {
^
/tmp/timer_createft0sob19.c: In function 'main':
/tmp/timer_createft0sob19.c:2:5: warning: implicit declaration of function 'timer_create' [-Wimplicit-function-declaration]
timer_create();
^
/tmp/timer_createft0sob19.c:3:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE build/temp.linux-aarch64-3.7/scratch/tmp/timer_createft0sob19.o -o build/temp.linux-aarch64-3.7/scratch/a.out
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -I/home/xys/projects/crossVenv4Py/cross/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-aarch64-3.7/scratch/vers.c -o build/temp.linux-aarch64-3.7/scratch/vers.o
build/temp.linux-aarch64-3.7/scratch/vers.c:4:17: fatal error: zmq.h: No such file or directory
#include "zmq.h"
^
compilation terminated.
error: command 'aarch64-linux-android-gcc' failed with exit status 1
Failed with default libzmq, trying again with /usr/local
{'libraries': ['zmq'], 'include_dirs': ['/usr/local/include'], 'library_dirs': ['/usr/local/lib'], 'runtime_library_dirs': ['/usr/local/lib'], 'extra_link_args': []}
************************************************
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir: /usr/local
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -c /tmp/timer_createkip93uia.c -o build/temp.linux-aarch64-3.7/scratch/tmp/timer_createkip93uia.o
/tmp/timer_createkip93uia.c:1:1: warning: return type defaults to 'int' [-Wreturn-type]
main (int argc, char **argv) {
^
/tmp/timer_createkip93uia.c: In function 'main':
/tmp/timer_createkip93uia.c:2:5: warning: implicit declaration of function 'timer_create' [-Wimplicit-function-declaration]
timer_create();
^
/tmp/timer_createkip93uia.c:3:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
Assembler messages:
Fatal error: can't create build/temp.linux-aarch64-3.7/scratch/tmp/timer_createkip93uia.o: No such file or directory
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -I/usr/local/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-aarch64-3.7/scratch/vers.c -o build/temp.linux-aarch64-3.7/scratch/vers.o
build/temp.linux-aarch64-3.7/scratch/vers.c:4:17: fatal error: zmq.h: No such file or directory
#include "zmq.h"
^
compilation terminated.
error: command 'aarch64-linux-android-gcc' failed with exit status 1
************************************************
Warning: Couldn't find an acceptable libzmq on the system.
If you expected pyzmq to link against an installed libzmq, please check to make sure:
* You have a C compiler installed
* A development version of Python is installed (including headers)
* A development version of ZMQ >= 3.2 is installed (including headers)
* If ZMQ is not in a default location, supply the argument --zmq=<path>
* If you did recently install ZMQ to a default location,
try rebuilding the ld cache with `sudo ldconfig`
or specify zmq's location with `--zmq=/usr/local`
You can skip all this detection/waiting nonsense if you know
you want pyzmq to bundle libzmq as an extension by passing:
`--zmq=bundled`
I will now try to build libzmq as a Python extension
unless you interrupt me (^C) in the next 10 seconds...
1...
************************************************
Using bundled libzmq
already have bundled/zeromq
attempting ./configure to generate platform.hpp
Warning: failed to configure libzmq:
b'/bin/sh: ./configure: \xe6\xb2\xa1\xe6\x9c\x89\xe9\x82\xa3\xe4\xb8\xaa\xe6\x96\x87\xe4\xbb\xb6\xe6\x88\x96\xe7\x9b\xae\xe5\xbd\x95\n'
staging platform.hpp from: /tmp/pip-install-5p6a3wit/pyzmq/buildutils/include_linux
************************************************
checking for timer_create
creating build/temp.linux-aarch64-3.7/tmp
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -c /tmp/timer_createcpy2lkyb.c -o build/temp.linux-aarch64-3.7/tmp/timer_createcpy2lkyb.o
/tmp/timer_createcpy2lkyb.c:1:1: warning: return type defaults to 'int' [-Wreturn-type]
main (int argc, char **argv) {
^
/tmp/timer_createcpy2lkyb.c: In function 'main':
/tmp/timer_createcpy2lkyb.c:2:5: warning: implicit declaration of function 'timer_create' [-Wimplicit-function-declaration]
timer_create();
^
/tmp/timer_createcpy2lkyb.c:3:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE build/temp.linux-aarch64-3.7/tmp/timer_createcpy2lkyb.o -o build/temp.linux-aarch64-3.7/a.out
ok
************************************************
building 'zmq.libzmq' extension
creating build/temp.linux-aarch64-3.7/buildutils
creating build/temp.linux-aarch64-3.7/bundled
creating build/temp.linux-aarch64-3.7/bundled/zeromq
creating build/temp.linux-aarch64-3.7/bundled/zeromq/src
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -Ibundled/zeromq/include -Ibundled -I/home/xys/projects/crossVenv4Py/cross/include -I/home/xys/projects/Python-3.7.1/out/android-64/include/python3.7m -c buildutils/initlibzmq.c -o build/temp.linux-aarch64-3.7/buildutils/initlibzmq.o
aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -D__ANDROID_API__=21 -D__ANDROID_API__=21 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -Ibundled/zeromq/include -Ibundled -I/home/xys/projects/crossVenv4Py/cross/include -I/home/xys/projects/Python-3.7.1/out/android-64/include/python3.7m -c bundled/zeromq/src/poller_base.cpp -o build/temp.linux-aarch64-3.7/bundled/zeromq/src/poller_base.o
In file included from bundled/zeromq/src/poller_base.cpp:31:0:
bundled/zeromq/src/poller_base.hpp:33:15: fatal error: map: No such file or directory
#include <map>
^
compilation terminated.
error: command 'aarch64-linux-android-gcc' failed with exit status 1
----------------------------------------
Command "/home/xys/projects/crossVenv4Py/cross/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-5p6a3wit/pyzmq/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-g19xjnhf/install-record.txt --single-version-externally-managed --compile --install-headers /home/xys/projects/crossVenv4Py/cross/include/site/python3.7/pyzmq" failed with error code 1 in /tmp/pip-install-5p6a3wit/pyzmq/
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(cross) ➜ ~/projects/crossVenv4Py
For some python installations there are multiple _sysconfigdata*
files. (For eg: python packaged in conda)
In those cases, crossenv
gives an error as it can't decide which file to use.
It'd be great to have a way to give a _sysconfigdata*
file on the command line.
sys.platform in cross-python is that of the build python instead of the host python. Is it possible to fix that?
Platform is just 'L'...
We had some trouble cross-compiling with python3.8 here: conda-forge/eups-feedstock#25 and it appears that the problem is that sometimes _get_sysconfig_name()
may be called with arguments (specifically the system allows the check_exists
argument. (I don't know if the problem is with python3.8 specifically, or if this is just a coincidence).
A fix proposed here https://github.com/conda-forge/crossenv-feedstock/blob/e53608da2d70a5b72a9f1f5750331be5c017b9b8/recipe/0002-Fix-py38-support.patch is to add a dummy *args
so that crossenv returns the correct value and simply swallows any arguments.
As discussed in #3, when using clang, the -print-sysroot option fails when trying to automatically determine the sysroot, and the error message is completely unhelpful.
This should probably be a warning, not an error, since the sysroot is often unnecessary.
I'm using crossenv to cross-compile Python modules for QNX, using QNX's qcc cross-compiler driver. Unfortunately qcc doesn't support any of the flags in find_compiler_info() -- no --version, no -print-sysroot, and no -dumpmachine. As a workaround I'm just commenting out the call to find_compiler_info()
in the CrossEnvBuilder
constructor, after which everything works great, but I'd like to look for a more permanent solution than modifying the source code after installation.
What would be the best way to manage different compilers? An option to disable the sanity checks? Expanding the sanity test to test-compile a short program (similar to how autotools does) rather than rely on possibly-unsupported flags? I don't know if there's a truly universal way to check the sysroot or target type.
I Tried to cross compile the wheel for boost-histogram with corssenv using pip. The build is passing for python 3.6, 3.7 and 3.10 but failing for 3.8 and 3.9. Can you please help with this?
WARNING: Discarding file:///project. Command errored out with exit status 1: /opt/python/cp38-cp38-xc/cross/bin/python /tmp/pip-standalone-pip-w2m72i8b/env_pip.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-4xhpmzwr/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=42' wheel toml 'setuptools_scm>=4.1.2' Check the logs for full command output.
ERROR: Command errored out with exit status 1: /opt/python/cp38-cp38-xc/cross/bin/python /tmp/pip-standalone-pip-w2m72i8b/env_pip.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-4xhpmzwr/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=42' wheel toml 'setuptools_scm>=4.1.2' Check the logs for full command output.
WARNING: You are using pip version 21.1.1; however, version 21.2.4 is available.
You should consider upgrading via the '/opt/python/cp38-cp38-xc/cross/bin/python -m pip install --upgrade pip' command.
Processing /project
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
✕ 3.79s
Error: Command ['python', '-m', 'pip', 'wheel', PurePosixPath('/project'), '--wheel-dir=/tmp/cibuildwheel/built_wheel', '--no-deps'] failed with code 1.
Checking for common errors...
x64_server@x_server_embedded:~/boost-histogram$
https://github.com/messense/manylinux2014-cross-arm/runs/2194258442?check_suite_focus=true
error: linking with `cc` failed: exit code: 1
|
= note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,--eh-frame-hdr" "-L" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.0.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.1.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.10.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.11.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.12.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.13.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.14.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.15.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.2.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.3.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.4.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.5.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.6.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.7.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.8.rcgu.o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.build_script_build.98y7fziv-cgu.9.rcgu.o" "-o" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/build/syn-f5721b8c67d18e56/build_script_build-f5721b8c67d18e56.2a6x0kqhg8aetyro.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/__w/manylinux2014-cross-arm/manylinux2014-cross-arm/pyo3-test/target/release/deps" "-L" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-b1047a2788193be4.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-e8e399e2092f3b2a.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-c6b02b522c91c060.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-eaad965909c4200f.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-dca728ac8ed54d87.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-4b4efaa8c4601753.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-d36e07ed2cc56ee4.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-3855d04b1bb40ceb.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-58a065cdff5508a2.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-82aeeedcc8d1450a.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-5c4d6c9d7595f844.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-2642d2bccc00517a.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-760e48b3f782e7f5.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-d4ef8bf89076ed01.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-36070e2029bb21ea.rlib" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-3aaf8f932781f33e.rlib" "-Wl,--end-group" "/github/home/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-e2523c360f03ba10.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc"
= note: /usr/bin/ld: /usr/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot/usr/libexec/../lib/Scrt1.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /usr/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot/usr/libexec/../lib/Scrt1.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /usr/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot/usr/libexec/../lib/Scrt1.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /usr/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot/usr/libexec/../lib/Scrt1.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /usr/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot/usr/libexec/../lib/Scrt1.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
Running the cc
command directly works fine, removing LIBRARY_PATH
from cross/bin/python
also works.
The error message is:
➜ ~/projects python3.7 -m crossenv ~/projects/Python-3.7.1/out/android-64/bin/python3.7 venv
ERROR: [Errno 2] No such file or directory: 'aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE': 'aarch64-linux-android-gcc --sysroot=/opt/local/android-ndk-r10e/platforms/android-21/arch-arm64 -pie -fPIE'
Seems crossenv gets host_cc info by reading "CC" environment variable from build-python's sysconfig.
I'm trying to build scipy for android. My steps are:
export PATH=$ANDROID_NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin:$ANDROID_NDK:$ANDROID_NDK/tools:/usr/local/bin:/usr/bin:/bin:$PATH
CROSS_COMPILER_PREFIX="arm-linux-androideabi-"
export ARCH="armeabi"
export CC="${CROSS_COMPILER_PREFIX}gcc --sysroot=$ANDROID_SYSROOT -pie -fPIE"
export CXX="${CROSS_COMPILER_PREFIX}g++ --sysroot=$ANDROID_SYSROOT -pie -fPIE"
export CPP="${CROSS_COMPILER_PREFIX}gcc -E --sysroot=$ANDROID_SYSROOT -pie -fPIE"
export AS="${CROSS_COMPILER_PREFIX}as"
export LD="${CROSS_COMPILER_PREFIX}ld --sysroot=$ANDROID_SYSROOT -pie -fPIE"
export GDB="${CROSS_COMPILER_PREFIX}gdb"
export STRIP="${CROSS_COMPILER_PREFIX}strip"
export RANLIB="${CROSS_COMPILER_PREFIX}ranlib"
export OBJCOPY="${CROSS_COMPILER_PREFIX}objcopy"
export OBJDUMP="${CROSS_COMPILER_PREFIX}objdump"
export AR="${CROSS_COMPILER_PREFIX}ar"
export NM="${CROSS_COMPILER_PREFIX}nm"
export READELF="${CROSS_COMPILER_PREFIX}readelf"
export M4=m4
export TARGET_PREFIX=$CROSS_COMPILER_PREFIX
export CONFIG_SITE="config.site"
export CFLAGS="-D__ANDROID_API__=21"
export CXXFLAGS="-D__ANDROID_API__=21"
export CPPFLAGS="-D__ANDROID_API__=21"
The exported environment is scanned and can be printed using python -m sysconfig
on my Android device.
I'm trying to use crossenv with a python build for android via https://github.com/yan12125/python3-android . This builds python with clang from google's android ndk.
I get the following error:
$ venv-native-only/bin/python -m crossenv python3-android/build/target/python/usr/bin/python3.8m venv-cross
clang: error: unknown argument: '-print-sysroot'
clang: error: no input files
ERROR: Command '['/home/build/dev/python3-android/venv-cross/build/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' died with <Signals.SIGABRT: 6>.
Hi,
First off, this tool looks very promising. However, I've run into a few issues. First off,
(cross) ; pip install -v numpy
Traceback (most recent call last):
File "/home/ori/src/py/xcenv/cross/bin/pip", line 7, in <module>
from pip._internal import main
ModuleNotFoundError: No module named 'pip._internal'
I can use pip as a module to make this work:
python -m pip install -v numpy
I'm using Debian, with a self-built Python 3.6. I'm trying to cross compile for Android API 23 (I've added some minor patches to get nl_langinfo to work.)
Without -pthread
set when compiling, bad things happen, such as this:
ImportError: /usr/local/lib/python3.8/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-38-arm-linux-gnueabi.so: undefined symbol: pthread_atfork
On my native python (Fedora 32, Python 3.8):
>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'gcc -pthread',
'CXX': 'g++ -pthread',
'LDSHARED': 'gcc -pthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now '
'-g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g'}
However in the cross-python:
>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc',
'CXX': 'arm-frc2020-linux-gnueabi-c++',
'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -shared'}
Unfortunately, python's configure script runs a program to determine if -pthread is needed. However, there's a partial fix for this. If one adds ac_cv_pthread_is_default=no ac_cv_pthread=yes ac_cv_cxx_thread=yes
to the cross ./configure
argument, you get:
>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc -pthread',
'CXX': 'arm-frc2020-linux-gnueabi-c++',
'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -pthread -shared'}
It seems like there isn't a way to override the CXX version without changing configure.ac (unless you have a good idea), filed an issue against CPython @ https://bugs.python.org/issue41916
Probably all/some of this discussion should be integrated into the cross-compilation documentation? Not sure how you'd like that to be done.
Really long story short, Our build framework starts by building a minimal python38 and includes the necessary wheels, including crossenv. It happens that all version >=1.0 of crossenv
makes PyNaCl
to fail to build in our cross-compiling environment, only for x86_64 architecture.
I presume that crossenv
has a bug that gets triggered in cross-compiling environment where if the target is of the same arch of the host it goes outside of its boundary instead of using the passed library paths of our toolchain.
Our PR where we hit the issue is: SynoCommunity/spksrc#4902
Issue opened on PyNaCl
(although problem seems to be with crossenv
): pyca/pynacl#688
Hi
I need the pycryptodome/lxml package for the following architecture of my target board. The target board does not have internet access so I cannot directly install pycryptodome/lxml on it. I need a method to compile it on my ubuntu 64 bit pc and then transfer it on my board. Following describes the toolchain I am using and the architecture of the target board. I cant find a .whl file or anything else for this architecture. Can I solve this problem using crossenv and how?
PPC_e5500_CC_BASE_DIR := /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux
ifeq (${TARGET_CPU}, PPC_e5500)
ifeq (${TARGET_CPU_BIT},)
$(info TARGET_CPU_BIT must be 32BIT/64BIT)
$(warning TARGET_CPU_BIT not specified, assuming 32BIT)
TARGET_CPU_BIT = 32BIT
endif
TARGET_ARCH = powerpc
COMPILER_TYPE = gcc
ifeq (${TARGET_CPU_BIT}, 32BIT)
#TARGET_CPU_BIT = 32BIT
CROSS_COMPILE_PATH = ${PPC_e5500_CC_BASE_DIR}/usr/bin/powerpc-fsl-linux:${PPC_e5500_CC_BASE_DIR}/usr/bin
CROSS_COMPILE = powerpc-fsl-linux-
HOST = powerpc-fsl-linux
CC_SYSROOT = ${PPC_e5500_CC_BASE_DIR}/../ppce5500-fsl-linux
COMMON_CFLAGS += -m32 -mhard-float -mcpu=e5500
COMMON_CFLAGS += --sysroot=${CC_SYSROOT}
Regards
Shubham Mishra
Some Python modules (notably Scipy and accelerated Numpy) require a Fortran compiler in addition to a C compiler. Inside crossenv the C compiler is set to whatever was used to compile host-python, but the Fortran compiler is still the system Fortran compiler, e.g. /usr/bin/gfortran rather than /usr/bin/arm-linux-armhf-gfortran. This means that the build will fail or produce an unusable module.
Is it ok to use the target and host provided openssl and zlib or is it known to cause problems?
Also is there an IRC or some other community?
PEP599 added non-x86 architectures to the manylinux spec, which have begun appearing on PyPI. These are based on CentOS7 but may work for some embedded systems using glibc and with the appropriate system libraries. Uses should be able to specify which manylinux platform tags are acceptable for their system.
Related #61
python -m crossenv /home/lx/platform3798C/pub/hi3798cv200/rootbox/usr/bin/python3.5 hisi_python --system-site-packages
Traceback (most recent call last):
File "/usr/lib/python3.5/runpy.py", line 174, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib/python3.5/runpy.py", line 133, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/usr/lib/python3.5/runpy.py", line 109, in _get_module_details
__import__(pkg_name)
File "/usr/lib/python3/dist-packages/pip/__init__.py", line 14, in <module>
from pip.utils import get_installed_distributions, get_prog
File "/usr/lib/python3/dist-packages/pip/utils/__init__.py", line 30, in <module>
from pip._vendor.retrying import retry
ImportError: No module named 'pip._vendor.retrying'
ERROR: Command '['/home/lx/hisi_python/cross/bin/python', '-m', 'pip', '--disable-pip-version-check', 'install', '--ignore-installed', '--prefix=/home/lx/hisi_python/cross', 'pip==8.1.1', 'setuptools==20.7.0']' returned non-zero exit status 1
the cross/bin/python is a py file, that convert PATH between BUILD and HOST, I tested with test.py
import sys
print(sys.path)
it works. But when I use crosspython as a Interpreter and just import pip, the ImportError message was the same. And crosspython -m python
error in the same way.
This would be a much better way to find breaking changes before they are released.
Ditto for setuptools, pip, et al., in separate jobs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.