Building wheel for horovod (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/jkoo/anaconda3/envs/dh/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9o1ukjdt/horovod/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9o1ukjdt/horovod/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-3c8nukd0
cwd: /tmp/pip-install-9o1ukjdt/horovod/
Complete output (283 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/horovod
copying horovod/__init__.py -> build/lib.linux-x86_64-3.7/horovod
creating build/lib.linux-x86_64-3.7/horovod/_keras
copying horovod/_keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/_keras
copying horovod/_keras/callbacks.py -> build/lib.linux-x86_64-3.7/horovod/_keras
creating build/lib.linux-x86_64-3.7/horovod/common
copying horovod/common/__init__.py -> build/lib.linux-x86_64-3.7/horovod/common
copying horovod/common/basics.py -> build/lib.linux-x86_64-3.7/horovod/common
copying horovod/common/util.py -> build/lib.linux-x86_64-3.7/horovod/common
creating build/lib.linux-x86_64-3.7/horovod/keras
copying horovod/keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/keras
copying horovod/keras/callbacks.py -> build/lib.linux-x86_64-3.7/horovod/keras
creating build/lib.linux-x86_64-3.7/horovod/mxnet
copying horovod/mxnet/__init__.py -> build/lib.linux-x86_64-3.7/horovod/mxnet
copying horovod/mxnet/mpi_ops.py -> build/lib.linux-x86_64-3.7/horovod/mxnet
creating build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/gloo_run.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/js_run.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/mpi_run.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/run_task.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/runner.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/task_fn.py -> build/lib.linux-x86_64-3.7/horovod/run
creating build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/gloo_run.py -> build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/mpi_run.py -> build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/runner.py -> build/lib.linux-x86_64-3.7/horovod/spark
creating build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/__init__.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/compression.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/mpi_ops.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/util.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
creating build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/__init__.py -> build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/compression.py -> build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/mpi_ops.py -> build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/sync_batch_norm.py -> build/lib.linux-x86_64-3.7/horovod/torch
creating build/lib.linux-x86_64-3.7/horovod/run/common
copying horovod/run/common/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/common
creating build/lib.linux-x86_64-3.7/horovod/run/driver
copying horovod/run/driver/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/driver
copying horovod/run/driver/driver_service.py -> build/lib.linux-x86_64-3.7/horovod/run/driver
creating build/lib.linux-x86_64-3.7/horovod/run/http
copying horovod/run/http/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/http
copying horovod/run/http/http_client.py -> build/lib.linux-x86_64-3.7/horovod/run/http
copying horovod/run/http/http_server.py -> build/lib.linux-x86_64-3.7/horovod/run/http
creating build/lib.linux-x86_64-3.7/horovod/run/task
copying horovod/run/task/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/task
copying horovod/run/task/task_service.py -> build/lib.linux-x86_64-3.7/horovod/run/task
creating build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/cache.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/lsf.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/network.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/threads.py -> build/lib.linux-x86_64-3.7/horovod/run/util
creating build/lib.linux-x86_64-3.7/horovod/run/common/service
copying horovod/run/common/service/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/common/service
copying horovod/run/common/service/driver_service.py -> build/lib.linux-x86_64-3.7/horovod/run/common/service
copying horovod/run/common/service/task_service.py -> build/lib.linux-x86_64-3.7/horovod/run/common/service
creating build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/codec.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/config_parser.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/env.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/host_hash.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/network.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/safe_shell_exec.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/secret.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/settings.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/timeout.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/tiny_shell_exec.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
creating build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/_namedtuple_fix.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/backend.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/cache.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/constants.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/estimator.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/params.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/serialization.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/store.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/util.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
creating build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/driver_service.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/job_id.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/mpirun_rsh.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/rsh.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
creating build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/bare.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/estimator.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/optimizer.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/remote.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/tensorflow.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/util.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
creating build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/gloo_exec_fn.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/mpirun_exec_fn.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/task_info.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/task_service.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
creating build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/estimator.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/remote.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/util.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
creating build/lib.linux-x86_64-3.7/horovod/tensorflow/keras
copying horovod/tensorflow/keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow/keras
copying horovod/tensorflow/keras/callbacks.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow/keras
creating build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib
copying horovod/torch/mpi_lib/__init__.py -> build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib
creating build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib_impl
copying horovod/torch/mpi_lib_impl/__init__.py -> build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib_impl
running build_ext
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -std=c++11 -fPIC -O3 -Wall -fassociative-math -ffast-math -ftree-vectorize -funsafe-math-optimizations -mavx -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.cc -o build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.o -o build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.so
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c build/temp.linux-x86_64-3.7/test_compile/test_link_flags.cc -o build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,--version-script=horovod.lds build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o -o build/temp.linux-x86_64-3.7/test_compile/test_link_flags.so
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 341, in get_mpi_flags
shlex.split(show_command), universal_newlines=True).strip()
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 488, in run
with Popen(*popenargs, **kwargs) as process:
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mpicxx': 'mpicxx'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 622, in get_common_options
mpi_flags = get_mpi_flags()
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 354, in get_mpi_flags
'%s' % (show_command, traceback.format_exc()))
distutils.errors.DistutilsPlatformError: mpicxx -show failed (see error below), is MPI in $PATH?
Note: If your version of MPI has a custom command to show compilation flags, please specify it with the HOROVOD_MPICXX_SHOW environment variable.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 341, in get_mpi_flags
shlex.split(show_command), universal_newlines=True).strip()
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 488, in run
with Popen(*popenargs, **kwargs) as process:
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mpicxx': 'mpicxx'
INFO: Cannot find MPI compilation flags, will skip compiling with MPI.
INFO: Unable to build TensorFlow plugin, will skip it.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 75, in check_tf_version
import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1526, in build_extensions
build_tf_extension(self, options)
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 933, in build_tf_extension
check_tf_version()
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 82, in check_tf_version
'import tensorflow failed, is it installed?\n\n%s' % traceback.format_exc())
distutils.errors.DistutilsPlatformError: import tensorflow failed, is it installed?
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 75, in check_tf_version
import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'
INFO: Unable to build PyTorch plugin, will skip it.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1134, in check_torch_version
import torch
ModuleNotFoundError: No module named 'torch'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1538, in build_extensions
torch_version = check_torch_version()
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1141, in check_torch_version
'import torch failed, is it installed?\n\n%s' % traceback.format_exc())
distutils.errors.DistutilsPlatformError: import torch failed, is it installed?
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1134, in check_torch_version
import torch
ModuleNotFoundError: No module named 'torch'
-- The CXX compiler identification is GNU 4.8.5
-- The C compiler identification is GNU 4.8.5
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/pip-install-9o1ukjdt/horovod/build/temp.linux-x86_64-3.7/gloo/mxnet
Scanning dependencies of target gloo
[ 6%] [ 6%] [ 9%] [ 12%] Building CXX object gloo/CMakeFiles/gloo.dir/allgather.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/allreduce.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/allgatherv.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/algorithm.cc.o
[ 16%] Building CXX object gloo/CMakeFiles/gloo.dir/allreduce_local.cc.o
[ 19%] Building CXX object gloo/CMakeFiles/gloo.dir/barrier.cc.o
[ 22%] Building CXX object gloo/CMakeFiles/gloo.dir/broadcast.cc.o
[ 25%] Building CXX object gloo/CMakeFiles/gloo.dir/context.cc.o
[ 29%] Building CXX object gloo/CMakeFiles/gloo.dir/gather.cc.o
[ 32%] Building CXX object gloo/CMakeFiles/gloo.dir/reduce.cc.o
[ 35%] Building CXX object gloo/CMakeFiles/gloo.dir/scatter.cc.o
[ 38%] Building CXX object gloo/CMakeFiles/gloo.dir/types.cc.o
[ 41%] Building CXX object gloo/CMakeFiles/gloo.dir/common/logging.cc.o
[ 45%] Building CXX object gloo/CMakeFiles/gloo.dir/common/linux.cc.o
[ 48%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/context.cc.o
[ 51%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/file_store.cc.o
[ 54%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/hash_store.cc.o
[ 58%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/prefix_store.cc.o
[ 61%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/store.cc.o
[ 64%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/address.cc.o
[ 67%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/buffer.cc.o
[ 70%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/context.cc.o
[ 74%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/device.cc.o
[ 77%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/pair.cc.o
[ 80%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/unbound_buffer.cc.o
[ 83%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/address.cc.o
[ 87%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/buffer.cc.o
[ 90%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/context.cc.o
[ 93%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/device.cc.o
[ 96%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/pair.cc.o
[100%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/unbound_buffer.cc.o
Linking CXX static library /tmp/pip-install-9o1ukjdt/horovod/build/temp.linux-x86_64-3.7/lib/mxnet/libgloo.a
[100%] Built target gloo
INFO: Unable to build MXNet plugin, will skip it.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 91, in check_mx_version
import mxnet as mx
ModuleNotFoundError: No module named 'mxnet'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1554, in build_extensions
build_mx_extension(self, options)
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1082, in build_mx_extension
check_mx_version()
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 98, in check_mx_version
'import mxnet failed, is it installed?\n\n%s' % traceback.format_exc())
distutils.errors.DistutilsPlatformError: import mxnet failed, is it installed?
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 91, in check_mx_version
import mxnet as mx
ModuleNotFoundError: No module named 'mxnet'
error: None of TensorFlow, PyTorch, or MXNet plugins were built. See errors above.
----------------------------------------
ERROR: Failed building wheel for horovod
Running setup.py clean for horovod
Building wheel for mpi4py (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/jkoo/anaconda3/envs/dh/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9o1ukjdt/mpi4py/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9o1ukjdt/mpi4py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-2vbbxt4_
cwd: /tmp/pip-install-9o1ukjdt/mpi4py/
Complete output (126 lines):
running bdist_wheel
running build
running build_src
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/mpi4py
copying src/mpi4py/run.py -> build/lib.linux-x86_64-3.7/mpi4py
copying src/mpi4py/__main__.py -> build/lib.linux-x86_64-3.7/mpi4py
copying src/mpi4py/bench.py -> build/lib.linux-x86_64-3.7/mpi4py
copying src/mpi4py/__init__.py -> build/lib.linux-x86_64-3.7/mpi4py
creating build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/futures/__main__.py -> build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/futures/_base.py -> build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/futures/server.py -> build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/futures/pool.py -> build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/futures/_lib.py -> build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/futures/__init__.py -> build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/futures/aplus.py -> build/lib.linux-x86_64-3.7/mpi4py/futures
copying src/mpi4py/MPI.pxd -> build/lib.linux-x86_64-3.7/mpi4py
copying src/mpi4py/__init__.pxd -> build/lib.linux-x86_64-3.7/mpi4py
copying src/mpi4py/libmpi.pxd -> build/lib.linux-x86_64-3.7/mpi4py
creating build/lib.linux-x86_64-3.7/mpi4py/include
creating build/lib.linux-x86_64-3.7/mpi4py/include/mpi4py
copying src/mpi4py/include/mpi4py/mpi4py.MPI_api.h -> build/lib.linux-x86_64-3.7/mpi4py/include/mpi4py
copying src/mpi4py/include/mpi4py/mpi4py.MPI.h -> build/lib.linux-x86_64-3.7/mpi4py/include/mpi4py
copying src/mpi4py/include/mpi4py/mpi4py.h -> build/lib.linux-x86_64-3.7/mpi4py/include/mpi4py
copying src/mpi4py/include/mpi4py/mpi4py.i -> build/lib.linux-x86_64-3.7/mpi4py/include/mpi4py
copying src/mpi4py/include/mpi4py/mpi.pxi -> build/lib.linux-x86_64-3.7/mpi4py/include/mpi4py
running build_clib
MPI configuration: [mpi] from 'mpi.cfg'
checking for library 'lmpe' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -llmpe -o _configtest
/home/jkoo/anaconda3/envs/dh/compiler_compat/ld: cannot find -llmpe
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
building 'mpe' dylib library
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
creating build/temp.linux-x86_64-3.7/src/lib-pmpi
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.7/src/lib-pmpi/mpe.o
creating build/lib.linux-x86_64-3.7/mpi4py/lib-pmpi
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,--no-as-needed build/temp.linux-x86_64-3.7/src/lib-pmpi/mpe.o -o build/lib.linux-x86_64-3.7/mpi4py/lib-pmpi/libmpe.so
checking for library 'vt-mpi' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -lvt-mpi -o _configtest
/home/jkoo/anaconda3/envs/dh/compiler_compat/ld: cannot find -lvt-mpi
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
checking for library 'vt.mpi' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -lvt.mpi -o _configtest
/home/jkoo/anaconda3/envs/dh/compiler_compat/ld: cannot find -lvt.mpi
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
building 'vt' dylib library
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c src/lib-pmpi/vt.c -o build/temp.linux-x86_64-3.7/src/lib-pmpi/vt.o
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,--no-as-needed build/temp.linux-x86_64-3.7/src/lib-pmpi/vt.o -o build/lib.linux-x86_64-3.7/mpi4py/lib-pmpi/libvt.so
checking for library 'vt-mpi' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -lvt-mpi -o _configtest
/home/jkoo/anaconda3/envs/dh/compiler_compat/ld: cannot find -lvt-mpi
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
checking for library 'vt.mpi' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -lvt.mpi -o _configtest
/home/jkoo/anaconda3/envs/dh/compiler_compat/ld: cannot find -lvt.mpi
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
building 'vt-mpi' dylib library
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c src/lib-pmpi/vt-mpi.c -o build/temp.linux-x86_64-3.7/src/lib-pmpi/vt-mpi.o
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,--no-as-needed build/temp.linux-x86_64-3.7/src/lib-pmpi/vt-mpi.o -o build/lib.linux-x86_64-3.7/mpi4py/lib-pmpi/libvt-mpi.so
checking for library 'vt-hyb' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -lvt-hyb -o _configtest
/home/jkoo/anaconda3/envs/dh/compiler_compat/ld: cannot find -lvt-hyb
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
checking for library 'vt.ompi' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -lvt.ompi -o _configtest
/home/jkoo/anaconda3/envs/dh/compiler_compat/ld: cannot find -lvt.ompi
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
building 'vt-hyb' dylib library
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c src/lib-pmpi/vt-hyb.c -o build/temp.linux-x86_64-3.7/src/lib-pmpi/vt-hyb.o
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,--no-as-needed build/temp.linux-x86_64-3.7/src/lib-pmpi/vt-hyb.o -o build/lib.linux-x86_64-3.7/mpi4py/lib-pmpi/libvt-hyb.so
running build_ext
MPI configuration: [mpi] from 'mpi.cfg'
checking for dlopen() availability ...
checking for header 'dlfcn.h' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c _configtest.c -o _configtest.o
success!
removing: _configtest.c _configtest.o
success!
checking for library 'dl' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -Lbuild/temp.linux-x86_64-3.7 -ldl -o _configtest
success!
removing: _configtest.c _configtest.o _configtest
checking for function 'dlopen' ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c _configtest.c -o _configtest.o
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ _configtest.o -Lbuild/temp.linux-x86_64-3.7 -ldl -o _configtest
success!
removing: _configtest.c _configtest.o _configtest
building 'mpi4py.dl' extension
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DHAVE_DLFCN_H=1 -DHAVE_DLOPEN=1 -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c src/dynload.c -o build/temp.linux-x86_64-3.7/src/dynload.o
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/dynload.o -Lbuild/temp.linux-x86_64-3.7 -ldl -o build/lib.linux-x86_64-3.7/mpi4py/dl.cpython-37m-x86_64-linux-gnu.so
checking for MPI compile and link ...
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c _configtest.c -o _configtest.o
_configtest.c:2:17: fatal error: mpi.h: No such file or directory
#include <mpi.h>
^
compilation terminated.
failure.
removing: _configtest.c _configtest.o
error: Cannot compile MPI programs. Check your configuration!!!
----------------------------------------
ERROR: Failed building wheel for mpi4py
Running setup.py clean for mpi4py
Failed to build horovod mpi4py
DEPRECATION: Could not build wheels for horovod, mpi4py which do not use PEP 517. pip will fall back to legacy 'setup.py install' for these. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Installing collected packages: termcolor, six, absl-py, gast, numpy, keras-preprocessing, opt-einsum, astor, google-pasta, tensorflow-estimator, grpcio, h5py, keras-applications, wrapt, zipp, importlib-metadata, markdown, protobuf, werkzeug, tensorboard, tensorflow, scipy, joblib, threadpoolctl, scikit-learn, pyyaml, pyaml, dh-scikit-optimize, xgboost, deap, soupsieve, beautifulsoup4, google, opencensus-context, cachetools, pyasn1, pyasn1-modules, rsa, google-auth, googleapis-common-protos, idna, chardet, urllib3, requests, pytz, google-api-core, opencensus, py-spy, filelock, prometheus-client, msgpack, colorful, psutil, nvidia-ml-py3, blessings, gpustat, multidict, attrs, typing-extensions, yarl, async-timeout, aiohttp, click, redis, pyrsistent, jsonschema, hiredis, aioredis, colorama, ray, cloudpickle, pycparser, cffi, horovod, decorator, networkx, python-dateutil, pandas, sphinxcontrib-serializinghtml, imagesize, alabaster, MarkupSafe, Jinja2, sphinxcontrib-applehelp, sphinxcontrib-htmlhelp, sphinxcontrib-devhelp, snowballstemmer, pyparsing, packaging, Pygments, docutils, babel, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinx, sphinx-rtd-theme, psycopg2-binary, mpi4py, sqlparse, asgiref, django, balsam-flow, pydot, keras, tqdm, cython, ConfigSpace, future, pyglet, gym, deephyper
Running setup.py install for horovod ... error
ERROR: Command errored out with exit status 1:
command: /home/jkoo/anaconda3/envs/dh/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9o1ukjdt/horovod/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9o1ukjdt/horovod/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-lznddl7n/install-record.txt --single-version-externally-managed --compile --install-headers /home/jkoo/anaconda3/envs/dh/include/python3.7m/horovod
cwd: /tmp/pip-install-9o1ukjdt/horovod/
Complete output (271 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/horovod
copying horovod/__init__.py -> build/lib.linux-x86_64-3.7/horovod
creating build/lib.linux-x86_64-3.7/horovod/_keras
copying horovod/_keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/_keras
copying horovod/_keras/callbacks.py -> build/lib.linux-x86_64-3.7/horovod/_keras
creating build/lib.linux-x86_64-3.7/horovod/common
copying horovod/common/__init__.py -> build/lib.linux-x86_64-3.7/horovod/common
copying horovod/common/basics.py -> build/lib.linux-x86_64-3.7/horovod/common
copying horovod/common/util.py -> build/lib.linux-x86_64-3.7/horovod/common
creating build/lib.linux-x86_64-3.7/horovod/keras
copying horovod/keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/keras
copying horovod/keras/callbacks.py -> build/lib.linux-x86_64-3.7/horovod/keras
creating build/lib.linux-x86_64-3.7/horovod/mxnet
copying horovod/mxnet/__init__.py -> build/lib.linux-x86_64-3.7/horovod/mxnet
copying horovod/mxnet/mpi_ops.py -> build/lib.linux-x86_64-3.7/horovod/mxnet
creating build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/gloo_run.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/js_run.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/mpi_run.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/run_task.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/runner.py -> build/lib.linux-x86_64-3.7/horovod/run
copying horovod/run/task_fn.py -> build/lib.linux-x86_64-3.7/horovod/run
creating build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/gloo_run.py -> build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/mpi_run.py -> build/lib.linux-x86_64-3.7/horovod/spark
copying horovod/spark/runner.py -> build/lib.linux-x86_64-3.7/horovod/spark
creating build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/__init__.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/compression.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/mpi_ops.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
copying horovod/tensorflow/util.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow
creating build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/__init__.py -> build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/compression.py -> build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/mpi_ops.py -> build/lib.linux-x86_64-3.7/horovod/torch
copying horovod/torch/sync_batch_norm.py -> build/lib.linux-x86_64-3.7/horovod/torch
creating build/lib.linux-x86_64-3.7/horovod/run/common
copying horovod/run/common/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/common
creating build/lib.linux-x86_64-3.7/horovod/run/driver
copying horovod/run/driver/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/driver
copying horovod/run/driver/driver_service.py -> build/lib.linux-x86_64-3.7/horovod/run/driver
creating build/lib.linux-x86_64-3.7/horovod/run/http
copying horovod/run/http/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/http
copying horovod/run/http/http_client.py -> build/lib.linux-x86_64-3.7/horovod/run/http
copying horovod/run/http/http_server.py -> build/lib.linux-x86_64-3.7/horovod/run/http
creating build/lib.linux-x86_64-3.7/horovod/run/task
copying horovod/run/task/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/task
copying horovod/run/task/task_service.py -> build/lib.linux-x86_64-3.7/horovod/run/task
creating build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/cache.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/lsf.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/network.py -> build/lib.linux-x86_64-3.7/horovod/run/util
copying horovod/run/util/threads.py -> build/lib.linux-x86_64-3.7/horovod/run/util
creating build/lib.linux-x86_64-3.7/horovod/run/common/service
copying horovod/run/common/service/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/common/service
copying horovod/run/common/service/driver_service.py -> build/lib.linux-x86_64-3.7/horovod/run/common/service
copying horovod/run/common/service/task_service.py -> build/lib.linux-x86_64-3.7/horovod/run/common/service
creating build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/__init__.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/codec.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/config_parser.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/env.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/host_hash.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/network.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/safe_shell_exec.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/secret.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/settings.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/timeout.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
copying horovod/run/common/util/tiny_shell_exec.py -> build/lib.linux-x86_64-3.7/horovod/run/common/util
creating build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/_namedtuple_fix.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/backend.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/cache.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/constants.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/estimator.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/params.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/serialization.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/store.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
copying horovod/spark/common/util.py -> build/lib.linux-x86_64-3.7/horovod/spark/common
creating build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/driver_service.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/job_id.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/mpirun_rsh.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
copying horovod/spark/driver/rsh.py -> build/lib.linux-x86_64-3.7/horovod/spark/driver
creating build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/bare.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/estimator.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/optimizer.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/remote.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/tensorflow.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
copying horovod/spark/keras/util.py -> build/lib.linux-x86_64-3.7/horovod/spark/keras
creating build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/gloo_exec_fn.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/mpirun_exec_fn.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/task_info.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
copying horovod/spark/task/task_service.py -> build/lib.linux-x86_64-3.7/horovod/spark/task
creating build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/__init__.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/estimator.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/remote.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
copying horovod/spark/torch/util.py -> build/lib.linux-x86_64-3.7/horovod/spark/torch
creating build/lib.linux-x86_64-3.7/horovod/tensorflow/keras
copying horovod/tensorflow/keras/__init__.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow/keras
copying horovod/tensorflow/keras/callbacks.py -> build/lib.linux-x86_64-3.7/horovod/tensorflow/keras
creating build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib
copying horovod/torch/mpi_lib/__init__.py -> build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib
creating build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib_impl
copying horovod/torch/mpi_lib_impl/__init__.py -> build/lib.linux-x86_64-3.7/horovod/torch/mpi_lib_impl
running build_ext
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -std=c++11 -fPIC -O3 -Wall -fassociative-math -ffast-math -ftree-vectorize -funsafe-math-optimizations -mavx -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.cc -o build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.o -o build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.so
gcc -pthread -B /home/jkoo/anaconda3/envs/dh/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jkoo/anaconda3/envs/dh/include/python3.7m -c build/temp.linux-x86_64-3.7/test_compile/test_link_flags.cc -o build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc -pthread -shared -B /home/jkoo/anaconda3/envs/dh/compiler_compat -L/home/jkoo/anaconda3/envs/dh/lib -Wl,-rpath=/home/jkoo/anaconda3/envs/dh/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,--version-script=horovod.lds build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o -o build/temp.linux-x86_64-3.7/test_compile/test_link_flags.so
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 341, in get_mpi_flags
shlex.split(show_command), universal_newlines=True).strip()
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 488, in run
with Popen(*popenargs, **kwargs) as process:
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mpicxx': 'mpicxx'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 622, in get_common_options
mpi_flags = get_mpi_flags()
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 354, in get_mpi_flags
'%s' % (show_command, traceback.format_exc()))
distutils.errors.DistutilsPlatformError: mpicxx -show failed (see error below), is MPI in $PATH?
Note: If your version of MPI has a custom command to show compilation flags, please specify it with the HOROVOD_MPICXX_SHOW environment variable.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 341, in get_mpi_flags
shlex.split(show_command), universal_newlines=True).strip()
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 488, in run
with Popen(*popenargs, **kwargs) as process:
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/home/jkoo/anaconda3/envs/dh/lib/python3.7/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mpicxx': 'mpicxx'
INFO: Cannot find MPI compilation flags, will skip compiling with MPI.
INFO: Compiler /usr/bin/g++ (version 4.8.5 20150623 (Red Hat 4.8.5-36)) is not usable for this TensorFlow installation. Require g++ (version >=7.3.1 20180303, <999).
INFO: Unable to build TensorFlow plugin, will skip it.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1526, in build_extensions
build_tf_extension(self, options)
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 989, in build_tf_extension
'Could not find compiler compatible with this TensorFlow installation.\n'
distutils.errors.DistutilsPlatformError: Could not find compiler compatible with this TensorFlow installation.
Please check the Horovod website for recommended compiler versions.
To force a specific compiler version, set CC and CXX environment variables.
INFO: Unable to build PyTorch plugin, will skip it.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1134, in check_torch_version
import torch
ModuleNotFoundError: No module named 'torch'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1538, in build_extensions
torch_version = check_torch_version()
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1141, in check_torch_version
'import torch failed, is it installed?\n\n%s' % traceback.format_exc())
distutils.errors.DistutilsPlatformError: import torch failed, is it installed?
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1134, in check_torch_version
import torch
ModuleNotFoundError: No module named 'torch'
-- The CXX compiler identification is GNU 4.8.5
-- The C compiler identification is GNU 4.8.5
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/pip-install-9o1ukjdt/horovod/build/temp.linux-x86_64-3.7/gloo/mxnet
Scanning dependencies of target gloo
[ 3%] [ 6%] [ 9%] [ 12%] Building CXX object gloo/CMakeFiles/gloo.dir/algorithm.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/allgather.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/allreduce.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/allgatherv.cc.o
[ 16%] Building CXX object gloo/CMakeFiles/gloo.dir/allreduce_local.cc.o
[ 19%] Building CXX object gloo/CMakeFiles/gloo.dir/barrier.cc.o
[ 22%] Building CXX object gloo/CMakeFiles/gloo.dir/broadcast.cc.o
[ 25%] Building CXX object gloo/CMakeFiles/gloo.dir/context.cc.o
[ 29%] Building CXX object gloo/CMakeFiles/gloo.dir/gather.cc.o
[ 32%] Building CXX object gloo/CMakeFiles/gloo.dir/reduce.cc.o
[ 35%] Building CXX object gloo/CMakeFiles/gloo.dir/scatter.cc.o
[ 38%] Building CXX object gloo/CMakeFiles/gloo.dir/types.cc.o
[ 41%] Building CXX object gloo/CMakeFiles/gloo.dir/common/logging.cc.o
[ 45%] Building CXX object gloo/CMakeFiles/gloo.dir/common/linux.cc.o
[ 48%] [ 51%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/context.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/file_store.cc.o
[ 54%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/hash_store.cc.o
[ 58%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/prefix_store.cc.o
[ 61%] Building CXX object gloo/CMakeFiles/gloo.dir/rendezvous/store.cc.o
[ 64%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/address.cc.o
[ 67%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/buffer.cc.o
[ 70%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/context.cc.o
[ 74%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/device.cc.o
[ 77%] [ 80%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/pair.cc.o
Building CXX object gloo/CMakeFiles/gloo.dir/transport/unbound_buffer.cc.o
[ 83%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/address.cc.o
[ 87%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/buffer.cc.o
[ 90%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/context.cc.o
[ 93%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/device.cc.o
[ 96%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/pair.cc.o
[100%] Building CXX object gloo/CMakeFiles/gloo.dir/transport/tcp/unbound_buffer.cc.o
Linking CXX static library /tmp/pip-install-9o1ukjdt/horovod/build/temp.linux-x86_64-3.7/lib/mxnet/libgloo.a
[100%] Built target gloo
INFO: Unable to build MXNet plugin, will skip it.
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 91, in check_mx_version
import mxnet as mx
ModuleNotFoundError: No module named 'mxnet'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1554, in build_extensions
build_mx_extension(self, options)
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 1082, in build_mx_extension
check_mx_version()
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 98, in check_mx_version
'import mxnet failed, is it installed?\n\n%s' % traceback.format_exc())
distutils.errors.DistutilsPlatformError: import mxnet failed, is it installed?
Traceback (most recent call last):
File "/tmp/pip-install-9o1ukjdt/horovod/setup.py", line 91, in check_mx_version
import mxnet as mx
ModuleNotFoundError: No module named 'mxnet'
error: None of TensorFlow, PyTorch, or MXNet plugins were built. See errors above.
----------------------------------------
ERROR: Command errored out with exit status 1: /home/jkoo/anaconda3/envs/dh/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9o1ukjdt/horovod/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9o1ukjdt/horovod/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-lznddl7n/install-record.txt --single-version-externally-managed --compile --install-headers /home/jkoo/anaconda3/envs/dh/include/python3.7m/horovod Check the logs for full command output.