tuxable-ltd / meta-scikit-learn Goto Github PK
View Code? Open in Web Editor NEWOpenEmbedded recipes for scikit-learn
License: MIT License
OpenEmbedded recipes for scikit-learn
License: MIT License
Hi, I am trying to build meta-scikit-learn with the following config:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "ubuntu-18.04"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "imx8mqevk"
DISTRO = "fsl-imx-wayland"
DISTRO_VERSION = "5.4-zeus"
TUNE_FEATURES = "aarch64 cortexa53 crc crypto"
TARGET_FPU = ""
meta
meta-poky = "HEAD:d88d62c20d7d8da85f02edb170dae0280624ad7e"
meta-oe
meta-multimedia
meta-python = "HEAD:2b5dd1eb81cd08bc065bc76125f2856e9383e98b"
meta-freescale = "HEAD:14f1a630a47375432f93c556927b879b51d84c4e"
meta-freescale-3rdparty = "HEAD:dbcc686f52c3c84db8cb86aa8973a4e373651b98"
meta-freescale-distro = "HEAD:ca27d12e4964d1336e662bcc60184bbff526c857"
meta-aws = "HEAD:4dcf78148d951339699b7650319fafda3ce0c3fc"
meta-virtualization = "zeus:48ec1b49871fef7d331ac1d0680a48d362c7482b"
meta-scikit-learn = "WIP:39031e8e706bdb57e35d1b85a9fb6f7da3ae61b1"
meta-bsp
meta-sdk
meta-ml = "HEAD:c4620c1b711d180a3f93d69ec6ca416d886f9277"
meta-nxp-demo-experience = "HEAD:c7263d9f3cc7bbf44e7164ffeda494cf283d3dec"
meta-browser = "HEAD:ee3be3b5986a4aa0e73df2204a625ae1fe5df37e"
meta-rust = "HEAD:53bfa324891966a2daf5d36dc13d4a43725aebed"
meta-clang = "HEAD:711e593d5984aad3bf35c51b7ac4482982bc16c7"
meta-gnome
meta-networking
meta-filesystems = "HEAD:2b5dd1eb81cd08bc065bc76125f2856e9383e98b"
meta-qt5 = "HEAD:21ce4c124d9a972d9122f87c64ac2773bf04c284"
meta-python2 = "HEAD:4400f9155ec193d028208cf0c66aeed2ba2b00ab"
I know this package is supposed to be supported on gatesgarth but I tried zeus just for my purposes. Ran into the following error during compilation:
ERROR: Task do_populate_sdk in /home/ubuntu/environment/imx-yocto-bsp/sources/meta-imx/meta-sdk/dynamic-layers/qt5-layer/recipes-fsl/images/imx-image-full.bb rdepends upon non-existent task do_package_write_deb in /home/ubuntu/environment/imx-yocto-bsp/sources/meta-scikit-learn/recipes-devtools/python/python3-scipy-native_1.5.2.bb
ERROR: Command execution failed: 1
Any hints on what to check?
Is it possible to use this layer with the Zeus version? Thank you.
I have attempted the following zeus build with both meta-scipy and meta-scikit-learn layers included but it failed with the following errors. Renaming python3-scipy_1.3%.bbappend to python3-scipy-native_1.3%.bbappend seemed to fix it. I am not sure what caused the failure in the first place though?
Build Configuration:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "ubuntu-18.04"
TARGET_SYS = "arm-poky-linux-gnueabi"
MACHINE = "imx6ul-var-dart"
DISTRO = "fsl-imx-x11"
DISTRO_VERSION = "5.4-zeus"
TUNE_FEATURES = "arm vfp cortexa7 neon thumb callconvention-hard"
TARGET_FPU = "hard"
meta
meta-poky = "HEAD:0275e68b0576cdefc19718e4b1977044f9f43f48"
meta-oe
meta-multimedia
meta-python = "HEAD:aad5b3d070cd8c58828b0975cf861d8ebc90f460"
meta-freescale = "HEAD:0be39721cea6e04c589a419f4bb07fabd0da6704"
meta-freescale-3rdparty = "HEAD:aea3771baa77e74762358ceb673d407e36637e5f"
meta-freescale-distro = "HEAD:ca27d12e4964d1336e662bcc60184bbff526c857"
meta-bsp
meta-sdk
meta-ml = "HEAD:64918e055f82e7c1d622a28856fb7ca1d1b93b47"
meta-browser = "HEAD:5f365ef0f842ba4651efe88787cf9c63bc8b6cb3"
meta-rust = "HEAD:5cda04c7c2158fc982502ca1e7893f71e4ca1729"
meta-gnome
meta-networking
meta-filesystems = "HEAD:aad5b3d070cd8c58828b0975cf861d8ebc90f460"
meta-qt5 = "HEAD:432ad2aa6c3a13253fefc909faba368851d21fb1"
meta-swupdate = "HEAD:8ad6a75e1778cdda671b3d6aea2a9692613020ec"
meta-virtualization = "HEAD:514ce5f6b240600f06b9956737be1e33900bacac"
meta-variscite-imx = "HEAD:037ae3cab30fcaeb17d465ff41efcb68c0a21a22"
meta-scipy = "zeus:757b20376ed7cb8e4497c6b9bdc5444a5b5a4e81"
meta-scikit-learn = "zeus:d2efaea114efbad76450e130c6c81df50561e565"
Initialising tasks: 100% |#######################################| Time: 0:00:01
Sstate summary: Wanted 8 Found 7 Missed 1 Current 418 (87% match, 99% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
ERROR: python3-scipy-1.3.3-r0 do_package_qa: QA Issue: /usr/lib/python3.7/site-packages/scipy/optimize/_trlib/_trlib.cpython-37m-arm-linux-gnueabi.so contained in package python3-scipy requires libblas.so.3, but no providers found in RDEPENDS_python3-scipy? [file-rdeps]
ERROR: python3-scipy-1.3.3-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
ERROR: Logfile of failure stored in: /home/jhuang/var-fsl-yocto/build_x11/tmp/work/cortexa7t2hf-neon-poky-linux-gnueabi/python3-scipy/1.3.3-r0/temp/log.do_package_qa.4163
ERROR: Task (/home/jhuang/var-fsl-yocto/sources/meta-scipy/recipes-devtools/python/python3-scipy_1.3.3.bb:do_package_qa) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1696 tasks of which 1695 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/home/jhuang/var-fsl-yocto/sources/meta-scipy/recipes-devtools/python/python3-scipy_1.3.3.bb:do_package_qa
Summary: There was 1 WARNING message shown.
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
I'm confused under what license this layer is distributed, would be happy to see LICENSE file as in other your repos
I'm using dunfell and when bitbaking I get an QA issue
ERROR: python3-scipy-1.3.3-r0 do_package_qa: QA Issue: /usr/lib/python3.8/site-packages/scipy/integrate/vode.cpython-38-aarch64-linux-gnu.so contained in package python3-scipy requires libblas.so.3()(64bit), but no providers found in RDEPENDS_python3-scipy? [file-rdeps]
ERROR: python3-scipy-1.3.3-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
When I'm not including meta-scikit-learn bitbake python3-scipy
is running without error.
Does anyone know how to proceed here?
Some function of scikit-learn fail on an arm 32 bit system. To reproduce build a scitkit-learn enabled image for arm, an example conf looks like:
Build Configuration:
BB_VERSION = "1.49.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "arch"
TARGET_SYS = "arm-oe-linux-gnueabi"
MACHINE = "qemuarm"
DISTRO = "nodistro"
DISTRO_VERSION = "nodistro.0"
TUNE_FEATURES = "arm armv7ve vfp thumb neon callconvention-hard"
TARGET_FPU = "hard"
meta = "master:68f20ac5524a5db8a09a90df796b4862cf747d3c"
meta-oe
meta-python = "master:759058bfb832161eb0d21f5c51a49c8361e727bc"
meta-scikit-learn = "<unknown>:<unknown>"
local.conf additions
MACHINE ?= "qemuarm"
IMAGE_INSTALL_append = "python3-scikit-learn"
EXTRA_IMAGE_FEATURES_append = " ssh-server-dropbear"
then
bitbake core-image-minimal
then
runqemu
once booted you can SSH into the qemu with
then run the pytest for the dbscan_inner
pytest /usr/lib/python3.9/site-packages/sklearn/cluster/tests/test_dbscan.py
which will output the following error
================================================ test session starts =================================================
platform linux -- Python 3.9.1, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /usr/lib/python3.9/site-packages/sklearn/cluster
collected 28 items
../../usr/lib/python3.9/site-packages/sklearn/cluster/tests/test_dbscan.py .FFFFatal Python error: Segmentation fault
Current thread 0xb6f76010 (most recent call first):
File "/usr/lib/python3.9/site-packages/sklearn/neighbors/_base.py", line 503 in _fit
File "/usr/lib/python3.9/site-packages/sklearn/neighbors/_unsupervised.py", line 142 in fit
File "/usr/lib/python3.9/site-packages/sklearn/cluster/tests/test_dbscan.py", line 85 in test_dbscan_sparse_precomputed
File "/usr/lib/python3.9/site-packages/_pytest/python.py", line 183 in pytest_pyfunc_call
File "/usr/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/lib/python3.9/site-packages/_pytest/python.py", line 1641 in runtest
File "/usr/lib/python3.9/site-packages/_pytest/runner.py", line 162 in pytest_runtest_call
File "/usr/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/lib/python3.9/site-packages/_pytest/runner.py", line 255 in <lambda>
File "/usr/lib/python3.9/site-packages/_pytest/runner.py", line 311 in from_call
File "/usr/lib/python3.9/site-packages/_pytest/runner.py", line 254 in call_runtest_hook
File "/usr/lib/python3.9/site-packages/_pytest/runner.py", line 215 in call_and_report
File "/usr/lib/python3.9/site-packages/_pytest/runner.py", line 126 in runtestprotocol
File "/usr/lib/python3.9/site-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
File "/usr/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/lib/python3.9/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
File "/usr/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/lib/python3.9/site-packages/_pytest/main.py", line 323 in _main
File "/usr/lib/python3.9/site-packages/_pytest/main.py", line 269 in wrap_session
File "/usr/lib/python3.9/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
File "/usr/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
File "/usr/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/lib/python3.9/site-packages/_pytest/config/__init__.py", line 162 in main
File "/usr/lib/python3.9/site-packages/_pytest/config/__init__.py", line 185 in console_main
File "/usr/bin/pytest", line 33 in <module>
I have identified this as a 64bit -> 32bit compile issue, probably from using the -native include files when building. The exact error from our proprietary codebase looks like:
File "sklearn/cluster/_dbscan_inner.pyx", line 40, in sklearn.cluster._dbscan_inner.dbscan_inner
ValueError: Buffer dtype mismatch, expected 'npy_intp' but got 'long long'
Which I think makes it obvious that it's using a long long
type as an integer pointer which would be valid on an 64bit system, but not on a 32bit system.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.