nixvital / ml-pkgs Goto Github PK
View Code? Open in Web Editor NEWExtra package for nix on machine learning and data science
License: MIT License
Extra package for nix on machine learning and data science
License: MIT License
I've tried pinning nixpkgs to the revision in this repo, as well as to 2d10e73416ec1449ef74aeac7faf2cf8c556ff5a, which is the same revision as when jaxlib was no longer marked as "fails to build" on this repo. I'm currently building against the latest stable nixpkgs, but I doubt it's going to work, since I've tried building it without this overlay, and no luck. This may be an upstream issue.
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/0sd34630xb88aca480v5wr8920s5pgnc-source
source root is source
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
patching script interpreter paths in ../output/external/org_tensorflow/third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc.tpl
../output/external/org_tensorflow/third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc.tpl: interpreter directive changed from "#!/usr/bin/env python" to "/nix/store/zdba9frlxj2ba8ca095win3nphsiiqhb-python3-3.10.8/bin/python"
Extracting Bazel installation...
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=0
INFO: Reading rc options for 'build' from /build/source/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /nix/store/xn6ql92hqwqh1yvvwnv6gmhi52hfpbnb-bazel-rc:
'build' options: --distdir=/nix/store/hcn6rxvvzig59nh7k3rmng8j809lv3ap-bazel-deps --extra_toolchains=@bazel_tools//tools/jdk:nonprebuilt_toolchain_definition --tool_java_runtime_version=local_jdk_11 --java_runtime_version=local_jdk_11
INFO: Reading rc options for 'build' from /build/source/.bazelrc:
'build' options: --apple_platform_type=macos --macos_minimum_os=10.14 --announce_rc --define open_source_build=true --spawn_strategy=standalone --enable_platform_specific_config --experimental_cc_shared_library --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_n>
INFO: Reading rc options for 'build' from /build/source/.jax_configure.bazelrc:
'build' options: --strategy=Genrule=standalone --repo_env PYTHON_BIN_PATH=/nix/store/zdba9frlxj2ba8ca095win3nphsiiqhb-python3-3.10.8/bin/python --action_env=PYENV_ROOT --python_path=/nix/store/zdba9frlxj2ba8ca095win3nphsiiqhb-python3-3.10.8/bin/python --distinct_host_configuration=false --define PROTOBUF_INCLUDE_PATH=/nix/store/3vm2ya1b52rdvjbansv39zdv0qwmhdcz-protobuf-3.20.>
INFO: Found applicable config definition build:short_logs in file /build/source/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:avx_posix in file /build/source/.bazelrc: --copt=-mavx --host_copt=-mavx
INFO: Found applicable config definition build:cuda in file /build/source/.bazelrc: --repo_env TF_NEED_CUDA=1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=sm_35,sm_52,sm_60,sm_70,compute_80 --crosstool_top=@local_config_cuda//crosstool:toolchain --@local_config_cuda//:enable_cuda --@org_tensorflow//tensorflow/compiler/xla/python:enable_gpu=true --define=xla_python_enable_gpu=true
INFO: Found applicable config definition build:cuda in file /build/source/.jax_configure.bazelrc: --action_env TF_CUDA_COMPUTE_CAPABILITIES=sm_35,sm_50,sm_60,sm_70,sm_75,compute_80
INFO: Found applicable config definition build:mkl_open_source_only in file /build/source/.bazelrc: --define=tensorflow_mkldnn_contraction_kernel=1
INFO: Found applicable config definition build:linux in file /build/source/.bazelrc: --config=posix --copt=-Wno-unknown-warning-option --copt=-Wno-stringop-truncation --copt=-Wno-array-parameter
INFO: Found applicable config definition build:posix in file /build/source/.bazelrc: --copt=-fvisibility=hidden --copt=-Wno-sign-compare --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
Loading:
Loading: 2 packages loaded
Analyzing: target //build:build_wheel (3 packages loaded, 0 targets configured)
Analyzing: target //build:build_wheel (46 packages loaded, 139 targets configured)
Analyzing: target //build:build_wheel (105 packages loaded, 855 targets configured)
Analyzing: target //build:build_wheel (215 packages loaded, 4629 targets configured)
Analyzing: target //build:build_wheel (235 packages loaded, 14022 targets configured)
INFO: Analyzed target //build:build_wheel (262 packages loaded, 15197 targets configured).
INFO: Found 1 target...
[0 / 60] [Prepa] Expanding template build/build_wheel ... (3 actions, 0 running)
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.devtools.build.lib.unsafe.StringUnsafe (file:/build/tmp/install/3cacfc6805558968a677264b92eb71de/A-server.jar) to constructor java.lang.String(byte[],byte)
WARNING: Please consider reporting this to the maintainers of com.google.devtools.build.lib.unsafe.StringUnsafe
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[25 / 96] Compiling tensorflow/compiler/xla/service/custom_call_status.cc; 1s local ... (16 actions, 15 running)
[47 / 109] Compiling absl/strings/internal/str_format/arg.cc; 3s local ... (16 actions, 15 running)
[78 / 133] Compiling absl/time/internal/cctz/src/time_zone_info.cc; 3s local ... (16 actions, 15 running)
[103 / 153] Compiling absl/synchronization/mutex.cc; 2s local ... (16 actions, 15 running)
[144 / 181] Compiling absl/strings/cord.cc; 3s local ... (16 actions, 15 running)
[240 / 1,622] Compiling jaxlib/ducc_fft.cc; 6s local ... (16 actions, 15 running)
[255 / 1,622] Compiling jaxlib/ducc_fft.cc; 9s local ... (16 actions, 15 running)
[270 / 1,622] Compiling jaxlib/ducc_fft_kernels.cc; 13s local ... (16 actions running)
[296 / 1,642] Compiling jaxlib/ducc_fft_kernels.cc; 17s local ... (16 actions, 15 running)
[362 / 1,690] Compiling jaxlib/ducc_fft_kernels.cc; 22s local ... (16 actions, 15 running)
[428 / 1,744] Compiling jaxlib/ducc_fft_kernels.cc; 29s local ... (16 actions, 15 running)
[518 / 1,816] Compiling jaxlib/ducc_fft_kernels.cc; 36s local ... (16 actions, 15 running)
ERROR: /build/output/external/local_config_cuda/cuda/BUILD:238:8: Executing genrule @local_config_cuda//cuda:cuda-include failed: (Exit 1): bash failed: error executing command /nix/store/dsd5gz46hdbdk2rfdimqddhq6m8m8fqs-bash-5.1-p16/bin/bash -c ... (remaining 1 argument skipped)
cp: cannot stat '/nix/store/xzr4m07l1ix0820xbn6jzb5297l13w1v-cudatoolkit-11.8.0-merged/include/./include': No such file or directory
Target //build:build_wheel failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /build/output/external/local_config_cuda/cuda/BUILD:50:20 Middleman _middlemen/@local_Uconfig_Ucuda_S_Scuda_Ccuda_Uheaders-BazelCppSemantics_build_arch_k8-opt failed: (Exit 1): bash failed: error executing command /nix/store/dsd5gz46hdbdk2rfdimqddhq6m8m8fqs-bash-5.1-p16/bin/bash -c ... (remaining 1 argument skipped)
INFO: Elapsed time: 57.694s, Critical Path: 37.63s
INFO: 547 processes: 159 internal, 388 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
It seems to me that the torch
has its own pybind11
. This means that if we have both torch
and pybind11
in our python environment, there will be a clash:
error: collision between `/nix/store/1wan18dqk50qs3a6mna4lnr3wlwh62xc-python3.10-torch-2.0.1-dev/include/pybind11/cast.h' and `/nix/store/m78gq5l1bnpb0q5q14pxbdcq37vpikix-python3.10-pybind11-2.10.4/include/pybind11/cast.h'
However the collision seems to be only about the header files, which is different from the python package clash.
This problem currently prevents having both pybind11
and torch
2.0.1 in the development environment, which is a headache.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.