ys7yoo/braincaffe2

Info on installing Caffe 2

BrainCaffe2

Info on building Caffe 2 from source

Install Caffe2 on Ubuntu (16.04 with GPUs)

Basically, follow official instruction here (with some modifications).

System libraries

sudo apt-get update
sudo apt-get install -y --no-install-recommends \
      build-essential \
      git \
      libgoogle-glog-dev \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      libprotobuf-dev \
      openmpi-bin \
      openmpi-doc \
      protobuf-compiler \
      libgflags-dev \
      cmake

Python packages

sudo pip3 install \
      future \
      numpy \
      protobuf \
      typing \
      hypothesis \
      pyyaml \
      pydot

Build from source

Under ~/src, run the following:

git clone https://github.com/pytorch/pytorch.git && cd pytorch
git checkout db5d313
git submodule update --init --recursive
CC=/usr/bin/gcc-5 CXX=/usr/bin/g++-5 python3 setup.py install

test

Check whether build was successful or not

python3 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

Test a module.

python3 caffe2/python/operator_test/activation_ops_test.py

Detectron installation

pip3 install -r requirements.txt  --user

Install Caffe2 on OSX (10.14 without GPUs)

Install Macports

• https://www.macports.org/install.php
• https://distfiles.macports.org/MacPorts/MacPorts-2.5.4-10.14-Mojave.pkg

Install systempackages using Macports

sudo port install cmake gflags

Set up python packages using conda

For python 2.7

conda create -n caffe

conda activate caffe

conda install future \
      numpy \
      protobuf \
      typing \
      hypothesis \
      pyyaml \
      pydot

For Python 3.6

conda create -n caffe36

conda activate caffe36

conda install future \
      numpy \
      protobuf \
      typing \
      hypothesis \
      pyyaml \
      pydot

Get source

git clone https://github.com/pytorch/pytorch.git && cd pytorch
git submodule update --init --recursive

Build and install

export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install

https://github.com/pytorch/pytorch#from-source

Set up path before launch python

export DYLD_LIBRARY_PATH=`python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"`/caffe2/python

After using it, you can unset DYLD_LIBRARY_PATH by

unset DYLD_LIBRARY_PATH

Alternatively, use Docker from Nvidia

braincaffe2's Issues

[OSX] copying locations

Copying extension caffe2.python.caffe2_pybind11_state
Copying caffe2.python.caffe2_pybind11_state from /Users/yyoo/src/pytorch/torch/lib/tmp_install/lib/python3.5/site-packages/caffe2/python/caffe2_pybind11_state.cpython-35m-darwin.so to /Users/yyoo/src/pytorch/build/lib.macosx-10.14-x86_64-3.5/caffe2/python/caffe2_pybind11_state.cpython-35m-darwin.so
copying /Users/yyoo/src/pytorch/torch/lib/tmp_install/lib/python3.5/site-packages/caffe2/python/caffe2_pybind11_state.cpython-35m-darwin.so -> /Users/yyoo/src/pytorch/build/lib.macosx-10.14-x86_64-3.5/caffe2/python
running install_lib
copying build/lib.macosx-10.14-x86_64-3.5/caffe2/proto/prof_dag_pb2.py -> /opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/caffe2/proto
error: could not delete '/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/caffe2/proto/prof_dag_pb2.py': Permission denied

[gamma] CUDA 9 is not compatible with GCC version >=6

-- Found CUDA: /usr/local/cuda (found version "9.0") 
-- Caffe2: CUDA detected: 9.0
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 9.0
-- Found CUDNN: /usr/local/cuda/include  
-- Found cuDNN: v7.0.5  (include: /usr/local/cuda/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so.7)
CMake Error at cmake/public/cuda.cmake:321 (message):
  CUDA 9.0 is not compatible with std::tuple from GCC version >= 6.  Please
  upgrade to CUDA 9.2 or use the following option to use another version (for
  example):

    -DCUDA_HOST_COMPILER=/usr/bin/gcc-5

Call Stack (most recent call first):
  cmake/Dependencies.cmake:698 (include)
  CMakeLists.txt:219 (include)

[alpha] Test failure

yyoo@alpha:~/src/pytorch$ python3 caffe2/python/operator_test/activation_ops_test.py
net_drawer will not run correctly. Please install the correct dependencies.

Try 1.0.0

Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
Failed to run 'bash ../tools/build_pytorch_libs.sh --use-cuda --use-nnpack --use-mkldnn --use-qnnpack caffe2'

[CMake] cannot find pthread

Determining if the pthread_create exist failed with the following output:
Change Dir: /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_61576/fast"
/usr/bin/make -f CMakeFiles/cmTC_61576.dir/build.make CMakeFiles/cmTC_61576.dir/build
make[1]: Entering directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_61576.dir/CheckSymbolExists.c.o
/usr/bin/cc    -fPIE   -o CMakeFiles/cmTC_61576.dir/CheckSymbolExists.c.o   -c /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c
Linking C executable cmTC_61576
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_61576.dir/link.txt --verbose=1
/usr/bin/cc        CMakeFiles/cmTC_61576.dir/CheckSymbolExists.c.o  -o cmTC_61576 -rdynamic 
CMakeFiles/cmTC_61576.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x1b): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_61576.dir/build.make:97: recipe for target 'cmTC_61576' failed
make[1]: *** [cmTC_61576] Error 1
make[1]: Leaving directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_61576/fast' failed
make: *** [cmTC_61576/fast] Error 2

File /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef pthread_create
  return ((int*)(&pthread_create))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_a007f/fast"
/usr/bin/make -f CMakeFiles/cmTC_a007f.dir/build.make CMakeFiles/cmTC_a007f.dir/build
make[1]: Entering directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_a007f.dir/CheckFunctionExists.c.o
/usr/bin/cc    -DCHECK_FUNCTION_EXISTS=pthread_create -fPIE   -o CMakeFiles/cmTC_a007f.dir/CheckFunctionExists.c.o   -c /usr/share/cmake-3.5/Modules/CheckFunctionExists.c
Linking C executable cmTC_a007f
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_a007f.dir/link.txt --verbose=1
/usr/bin/cc    -DCHECK_FUNCTION_EXISTS=pthread_create    CMakeFiles/cmTC_a007f.dir/CheckFunctionExists.c.o  -o cmTC_a007f -rdynamic -lpthreads 
/usr/bin/x86_64-linux-gnu-ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_a007f.dir/build.make:97: recipe for target 'cmTC_a007f' failed
make[1]: *** [cmTC_a007f] Error 1
make[1]: Leaving directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_a007f/fast' failed
make: *** [cmTC_a007f/fast] Error 2


Performing C SOURCE FILE Test BLAS_F2C_DOUBLE_WORKS failed with the following output:
Change Dir: /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_710f5/fast"
/usr/bin/make -f CMakeFiles/cmTC_710f5.dir/build.make CMakeFiles/cmTC_710f5.dir/build
make[1]: Entering directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_710f5.dir/src.c.o
/usr/bin/cc    -DBLAS_F2C_DOUBLE_WORKS   -o CMakeFiles/cmTC_710f5.dir/src.c.o   -c /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_710f5
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_710f5.dir/link.txt --verbose=1
/usr/bin/cc    -DBLAS_F2C_DOUBLE_WORKS   -rdynamic  CMakeFiles/cmTC_710f5.dir/src.c.o  -o cmTC_710f5 /usr/lib/libblas.so 
make[1]: Leaving directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'

Return value: 1
Source file was:

#include <stdlib.h>
#include <stdio.h>
float x[4] = { 1, 2, 3, 4 };
float y[4] = { .1, .01, .001, .0001 };
int four = 4;
int one = 1;
extern double sdot_();
int main() {
  int i;
  double r = sdot_(&four, x, &one, y, &one);
  exit((float)r != (float).1234);
}
Performing C++ SOURCE FILE Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 failed with the following output:
Change Dir: /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_3de2c/fast"
/usr/bin/make -f CMakeFiles/cmTC_3de2c.dir/build.make CMakeFiles/cmTC_3de2c.dir/build
make[1]: Entering directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_3de2c.dir/src.cxx.o
/usr/bin/c++     -fvisibility-inlines-hidden  -std=c++11  -Wall  -Wextra  -Wshadow  -pedantic  -pedantic-errors -DHAVE_CXX_FLAG_WSHORTEN_64_TO_32  -Wshorten-64-to-32   -Wshorten-64-to-32 -o CMakeFiles/cmTC_3de2c.dir/src.cxx.o -c /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp/src.cxx
c++: error: unrecognized command line option '-Wshorten-64-to-32'
c++: error: unrecognized command line option '-Wshorten-64-to-32'
CMakeFiles/cmTC_3de2c.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_3de2c.dir/src.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_3de2c.dir/src.cxx.o] Error 1
make[1]: Leaving directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_3de2c/fast' failed
make: *** [cmTC_3de2c/fast] Error 2

Source file was:
int main() { return 0; }
Performing C++ SOURCE FILE Test HAVE_CXX_FLAG_WD654 failed with the following output:
Change Dir: /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_74cd6/fast"
/usr/bin/make -f CMakeFiles/cmTC_74cd6.dir/build.make CMakeFiles/cmTC_74cd6.dir/build
make[1]: Entering directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_74cd6.dir/src.cxx.o
/usr/bin/c++     -fvisibility-inlines-hidden  -std=c++11  -Wall  -Wextra  -Wshadow  -pedantic  -pedantic-errors  -Wfloat-equal  -fstrict-aliasing  -Wno-deprecated-declarations  -Wstrict-aliasing -DHAVE_CXX_FLAG_WD654  -wd654   -wd654 -o CMakeFiles/cmTC_74cd6.dir/src.cxx.o -c /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp/src.cxx
c++: error: unrecognized command line option '-wd654'
c++: error: unrecognized command line option '-wd654'
CMakeFiles/cmTC_74cd6.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_74cd6.dir/src.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_74cd6.dir/src.cxx.o] Error 1
make[1]: Leaving directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_74cd6/fast' failed
make: *** [cmTC_74cd6/fast] Error 2

Source file was:
int main() { return 0; }
Performing C++ SOURCE FILE Test HAVE_CXX_FLAG_WTHREAD_SAFETY failed with the following output:
Change Dir: /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_c591c/fast"
/usr/bin/make -f CMakeFiles/cmTC_c591c.dir/build.make CMakeFiles/cmTC_c591c.dir/build
make[1]: Entering directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_c591c.dir/src.cxx.o
/usr/bin/c++     -fvisibility-inlines-hidden  -std=c++11  -Wall  -Wextra  -Wshadow  -pedantic  -pedantic-errors  -Wfloat-equal  -fstrict-aliasing  -Wno-deprecated-declarations  -Wstrict-aliasing -DHAVE_CXX_FLAG_WTHREAD_SAFETY  -Wthread-safety   -Wthread-safety -o CMakeFiles/cmTC_c591c.dir/src.cxx.o -c /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp/src.cxx
c++: error: unrecognized command line option '-Wthread-safety'; did you mean '-fthread-jumps'?
c++: error: unrecognized command line option '-Wthread-safety'; did you mean '-fthread-jumps'?
CMakeFiles/cmTC_c591c.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_c591c.dir/src.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_c591c.dir/src.cxx.o] Error 1
make[1]: Leaving directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_c591c/fast' failed
make: *** [cmTC_c591c/fast] Error 2

Source file was:
int main() { return 0; }
Performing C++ SOURCE FILE Test __CxxFlag__Winconsistent_missing_override failed with the following output:
Change Dir: /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_e84b6/fast"
/usr/bin/make -f CMakeFiles/cmTC_e84b6.dir/build.make CMakeFiles/cmTC_e84b6.dir/build
make[1]: Entering directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_e84b6.dir/src.cxx.o
/usr/bin/c++     -fvisibility-inlines-hidden -fopenmp -D__CxxFlag__Winconsistent_missing_override   -Winconsistent-missing-override -o CMakeFiles/cmTC_e84b6.dir/src.cxx.o -c /home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp/src.cxx
c++: error: unrecognized command line option '-Winconsistent-missing-override'; did you mean '-Wno-suggest-override'?
CMakeFiles/cmTC_e84b6.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_e84b6.dir/src.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_e84b6.dir/src.cxx.o] Error 1
make[1]: Leaving directory '/home/yyoo/src/pytorch/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_e84b6/fast' failed
make: *** [cmTC_e84b6/fast] Error 2

Source file was:
int main() { return 0; }

OpenCV issue?

Traceback (most recent call last):
  File "tools/infer_simple.py", line 185, in <module>
    main(args)
  File "tools/infer_simple.py", line 177, in main
    out_when_no_box=args.out_when_no_box
  File "/home/yyoo/src/detectron/detectron/utils/vis.py", line 332, in vis_one_image
    e.copy(), cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
ValueError: not enough values to unpack (expected 3, got 2)

[OSX] TLS Key?

(caffe37) MP4:~ yyoo$ python
Python 3.7.2 (default, Dec 29 2018, 00:00:04) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from torch.caffe2 import core
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda3/envs/caffe37/lib/python3.7/site-packages/torch/__init__.py", line 84, in <module>
    from torch._C import *
RuntimeError: get_internals: could not successfully initialize the TLS key!

[beta] linking pytorch.lib

[ 73%] Linking CXX shared library ../lib/libcaffe2_gpu.so
[ 79%] Built target caffe2_gpu
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
  File "setup.py", line 720, in <module>
    build_deps()
  File "setup.py", line 279, in build_deps
    build_dir='build')
  File "/home/yyoo/src/pytorch/tools/build_pytorch_libs.py", line 248, in build_caffe2
    check_call(['make', '-j', str(max_jobs), 'install'], cwd=build_dir, env=my_env)
  File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j', '12', 'install']' returned non-zero exit status 2

[OSX] numpy issue

MP4:pytorch yyoo$ python3
Python 3.5.6 (default, Oct  3 2018, 19:57:36) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from caffe2.python import core
Traceback (most recent call last):
  File "/Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/__init__.py", line 16, in <module>
    from . import multiarray
  File "/Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/overrides.py", line 6, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/_multiarray_umath.cpython-35m-darwin.so, 2): Symbol not found: ___addtf3
  Referenced from: /Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/../.dylibs/libquadmath.0.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in /Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/../.dylibs/libquadmath.0.dylib

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yyoo/src/pytorch/caffe2/python/core.py", line 15, in <module>
    from caffe2.python import scope, utils, workspace
  File "/Users/yyoo/src/pytorch/caffe2/python/utils.py", line 17, in <module>
    import numpy as np
  File "/Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/__init__.py", line 47, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
  your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
  1. Check that you are using the Python you expect (you're using /opt/local/bin/python3),
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy versions you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

     Note: this error has many possible causes, so please don't comment on
     an existing issue about this - open a new one instead.

Original error was: dlopen(/Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/_multiarray_umath.cpython-35m-darwin.so, 2): Symbol not found: ___addtf3
  Referenced from: /Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/../.dylibs/libquadmath.0.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in /Users/yyoo/Library/Python/3.5/lib/python/site-packages/numpy/core/../.dylibs/libquadmath.0.dylib

[gamma] external MKL locations

-- Found MKLDNN: /home/yyoo/src/pytorch/third_party/ideep/mkl-dnn/external/mklml_lnx_2018.0.3.20180406/lib/libmklml_intel.so;/usr/lib/x86_64-linux-gnu/libiomp5.so  
-- Detecting Intel(R) MKL: trying mklml_intel
-- Intel(R) MKL: include /home/yyoo/src/pytorch/third_party/ideep/mkl-dnn/external/mklml_lnx_2018.0.3.20180406/include
-- Intel(R) MKL: lib /home/yyoo/src/pytorch/third_party/ideep/mkl-dnn/external/mklml_lnx_2018.0.3.20180406/lib/libmklml_intel.so
-- Intel(R) MKL: OpenMP lib /home/yyoo/src/pytorch/third_party/ideep/mkl-dnn/external/mklml_lnx_2018.0.3.20180406/lib/libiomp5.so

[alpha] install log

[  0%] Built target pthreadpool
[  0%] Built target onnxifi_loader
[  1%] Built target gtest
[  2%] Built target fbgemm_avx2
[  2%] Built target fbgemm_avx512
[  2%] Built target clog
[  3%] Built target gloo
[  4%] Built target libprotobuf-lite
[  4%] Built target benchmark
[  4%] Built target fbgemm_generic
[  4%] Built target mkrename
[  5%] Built target asmjit
[  5%] Built target onnxifi_dummy
[  5%] Built target common
[  5%] Built target ATEN_CPU_FILES_GEN_TARGET
[  6%] Built target c10
[  6%] Built target ATEN_CUDA_FILES_GEN_TARGET
[  6%] Built target mkdisp
[  6%] Built target mkmasked_gnuabi
[  6%] Built target mkrename_gnuabi
[  6%] Built target mkalias
[  9%] Built target libprotobuf
[ 29%] Built target python_copy_files
[ 30%] Built target cpuinfo
[ 30%] Built target arraymap
[ 30%] Built target gtest_main
[ 30%] Built target nnpack_reference_layers
[ 31%] Built target cpuinfo_internals
[ 31%] Built target benchmark_main
[ 31%] Built target onnxifi_wrapper
[ 31%] Built target __aten_op_header_gen
[ 31%] Built target c10_cuda
[ 31%] Built target renameAVX.h_generated
[ 31%] Built target headers
[ 31%] Built target renamedsp128.h_generated
[ 31%] Built target dispsse.c_generated
[ 31%] Built target renameFMA4.h_generated
[ 31%] Built target renamedsp256.h_generated
[ 31%] Built target renameSSE2.h_generated
[ 31%] Built target dispavx.c_generated
[ 31%] Built target renameAVX2128.h_generated
[ 31%] Built target gloo_cuda
[ 31%] Built target renameAVX2.h_generated
[ 31%] Built target torch_python_stubs
[ 31%] Built target renameSSE4.h_generated
[ 31%] Built target fbgemm
[ 33%] Built target c10_InlineStreamGuard_test
[ 33%] Built target c10_StreamGuard_test
[ 33%] Built target c10_InlineDeviceGuard_test
[ 33%] Built target qnnpack
[ 34%] Built target nnpack
[ 34%] Built target c10_DeviceGuard_test
[ 34%] Built target c10_logging_test
[ 34%] Built target c10_TypeTraits_test
[ 34%] Built target c10_Array_test
[ 34%] Built target c10_intrusive_ptr_test
[ 34%] Built target c10_Half_test
[ 34%] Built target c10_Metaprogramming_test
[ 34%] Built target c10_flags_test
[ 34%] Built target c10_TypeList_test
[ 34%] Built target c10_cuda_CUDATest
[ 34%] Built target c10_registry_test
[ 34%] Built target c10_typeid_test
[ 34%] Built target sleefavx
[ 34%] Built target dispsse_obj
[ 34%] Built target dispavx_obj
[ 35%] Built target c10_TensorTypeId_test
[ 35%] Built target sleefavx2
[ 35%] Built target sleefsse2
[ 35%] Built target sleeffma4
[ 35%] Built target sleefsse4
[ 35%] Built target sleefavx2128
[ 35%] Built target gloo_builder
[ 38%] Built target libprotoc
[ 38%] Built target sleef
[ 38%] Built target protoc
[ 38%] Built target gen_onnx_proto
[ 38%] Built target Caffe2_PROTO
[ 38%] Built target onnx_proto
[ 38%] Built target Caffe2_perfkernels_avx512
[ 38%] Built target caffe2_protos
[ 38%] Built target Caffe2_perfkernels_avx
[ 38%] Built target caffe2_dnnlowp_avx2_ops
[ 39%] Built target Caffe2_perfkernels_avx2
[ 41%] Built target onnx
[ 64%] Built target caffe2
[ 64%] Built target caffe2_module_test_dynamic
[ 64%] Built target shm
[ 64%] Built target caffe2_observers
[ 64%] Built target caffe2_pybind11_state
[ 64%] Built target torch_shm_manager
[ 65%] Built target THD
[ 80%] Built target caffe2_gpu
[ 80%] Built target cuda_stream_test
[ 80%] Built target blob_gpu_test
[ 80%] Built target utility_ops_gpu_test
[ 80%] Built target operator_gpu_test
[ 80%] Built target cuda_apply_test
[ 80%] Built target operator_fallback_gpu_test
[ 80%] Built target cuda_tensor_interop_test
[ 80%] Built target cuda_cudnn_test
[ 81%] Built target generate_proposals_op_gpu_test
[ 81%] Built target batch_matmul_op_gpu_test
[ 81%] Built target mpi_gpu_test
[ 81%] Built target event_test
[ 81%] Built target roi_align_op_gpu_test
[ 81%] Built target undefined_tensor_test
[ 81%] Built target parallel_net_test
[ 81%] Built target pattern_net_transform_test
[ 81%] Built target graph_test
[ 81%] Built target event_gpu_test
[ 81%] Built target operator_test
[ 81%] Built target net_dag_utils_test
[ 81%] Built target net_async_tracing_test
[ 81%] Built target weakref_test
[ 81%] Built target tensor_interop_test
[ 81%] Built target TarjansImplTest
[ 82%] Built target native_test
[ 82%] Built target timer_test
[ 82%] Built target extension_backend_test
[ 82%] Built target half_test
[ 82%] Built target TopoSortTest
[ 82%] Built target reshape_op_gpu_test
[ 82%] Built target scalar_test
[ 82%] Built target cuda_packedtensoraccessor_test
[ 82%] Built target init_test
[ 82%] Built target cuda_half_test
[ 82%] Built target transform_test
[ 83%] Built target wrapdim_test
[ 83%] Built target TensorImpl_test
[ 83%] Built target int8_roi_align_op_test
[ 83%] Built target inline_container_test
[ 83%] Built target dlconvertor_test
[ 83%] Built target common_test
[ 83%] Built target conv_op_cache_cudnn_test
[ 83%] Built target workspace_test
[ 83%] Built target module_test
[ 83%] Built target smart_tensor_printer_test
[ 83%] Built target net_simple_refcount_test
[ 83%] Built target cast_test
[ 83%] Built target context_test
[ 83%] Built target tbb_init_test
[ 83%] Built target verify_api_visibility
[ 83%] Built target string_ops_test
[ 83%] Built target net_test
[ 84%] Built target device_test
[ 85%] Built target stats_test
[ 85%] Built target test_parallel
[ 85%] Built target atest
[ 85%] Built target apply_utils_test
[ 85%] Built target scalar_tensor_test
[ 85%] Built target basic
[ 85%] Built target int8_test
[ 85%] Built target observer_test
[ 85%] Built target math_test
[ 85%] Built target fatal_signal_asan_no_sig_test
[ 85%] Built target fixed_divisor_test
[ 85%] Built target context_gpu_test
[ 86%] Built target simple_queue_test
[ 86%] Built target generate_proposals_op_util_boxes_test
[ 86%] Built target mpi_test
[ 86%] Built target MatchTest
[ 86%] Built target text_file_reader_utils_test
[ 86%] Built target elementwise_op_gpu_test
[ 86%] Built target common_subexpression_elimination_test
[ 86%] Built target proto_utils_test
[ 86%] Built target SubgraphMatcherTest
[ 86%] Built target cuda_rng_test
[ 86%] Built target generate_proposals_op_util_nms_test
[ 86%] Built target math_gpu_test
[ 86%] Built target generate_proposals_op_util_nms_gpu_test
[ 86%] Built target NeuralNetTest
[ 86%] Built target GraphTest
[ 86%] Built target BinaryMatchImplTest
[ 86%] Built target cuda_integer_divider_test
[ 86%] Built target AlgorithmsTest
[ 86%] Built target depthwise3x3_conv_op_test
[ 87%] Built target caffe2_pybind11_state_gpu
[ 88%] Built target boolean_unmask_ops_test
[ 88%] Built target ssa_test
[ 88%] Built target time_observer_test
[ 88%] Built target elementwise_op_test
[ 88%] Built target blob_test
[ 88%] Built target operator_schema_test
[ 88%] Built target generate_proposals_op_test
[ 88%] Built target conv_transpose_op_mobile_test
[ 88%] Built target conv_to_nnpack_transform_test
[ 89%] Built target predictor_test
[ 89%] Built target batch_matmul_op_test
[ 89%] Built target mobile_test
[ 89%] Built target broadcast_test
[ 90%] Built target backend_cutting_test
[ 90%] Built target cpuid_test
[ 90%] Built target distributed_test
[ 90%] Built target bound_shape_inference_test
[ 90%] Built target utility_ops_test
[ 90%] Built target converter_nomigraph_test
[ 90%] Built target cuda_optional_test
[ 90%] Built target dead_code_elim_test
[ 90%] Built target nnpack_test
[ 90%] Built target net_gpu_test
[ 91%] Built target c10d
[ 91%] Built target TCPStoreTest
[ 91%] Built target ProcessGroupMPITest
[ 91%] Built target FileStoreTest
[ 91%] Built target c10d_cuda_test
[ 91%] Built target ProcessGroupNCCLTest
[ 91%] Built target ProcessGroupGlooAsyncTest
[ 91%] Built target ProcessGroupGlooTest
[ 92%] Built target caffe2_detectron_ops_gpu
[ 96%] Built target torch
[ 96%] Built target test_jit
[ 97%] Built target test_api
[100%] Built target torch_python
Install the project...
-- Install configuration: "Release"
Building wheel torch-1.1.0a0+98b333d
-- Building version 1.1.0a0+98b333d
running install
running build
running build_py
copying torch/version.py -> build/lib.linux-x86_64-3.5/torch
copying caffe2/proto/caffe2_legacy_pb2.py -> build/lib.linux-x86_64-3.5/caffe2/proto
copying caffe2/proto/metanet_pb2.py -> build/lib.linux-x86_64-3.5/caffe2/proto
copying caffe2/proto/prof_dag_pb2.py -> build/lib.linux-x86_64-3.5/caffe2/proto
copying caffe2/proto/torch_pb2.py -> build/lib.linux-x86_64-3.5/caffe2/proto
copying caffe2/proto/predictor_consts_pb2.py -> build/lib.linux-x86_64-3.5/caffe2/proto
copying caffe2/proto/caffe2_pb2.py -> build/lib.linux-x86_64-3.5/caffe2/proto
copying caffe2/proto/hsm_pb2.py -> build/lib.linux-x86_64-3.5/caffe2/proto
running build_ext
-- Building with NumPy bindings
-- Detected cuDNN at /usr/local/cuda/lib64/libcudnn.so.7, /usr/local/cuda/include
-- Not using MIOpen
-- Detected CUDA at /usr/local/cuda
-- Using MKLDNN
-- Using system provided NCCL library at /usr/lib/x86_64-linux-gnu/libnccl.so.2.2.13, /usr/include
-- Building with THD distributed package 
-- Building with c10d distributed package 

Copying extension caffe2.python.caffe2_pybind11_state
Copying caffe2.python.caffe2_pybind11_state from torch/lib/python3/dist-packages/caffe2/python/caffe2_pybind11_state.cpython-35m-x86_64-linux-gnu.so to /home/yyoo/src/pytorch/build/lib.linux-x86_64-3.5/caffe2/python/caffe2_pybind11_state.cpython-35m-x86_64-linux-gnu.so

Copying extension caffe2.python.caffe2_pybind11_state_gpu
Copying caffe2.python.caffe2_pybind11_state_gpu from torch/lib/python3/dist-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-35m-x86_64-linux-gnu.so to /home/yyoo/src/pytorch/build/lib.linux-x86_64-3.5/caffe2/python/caffe2_pybind11_state_gpu.cpython-35m-x86_64-linux-gnu.so
running install_lib
copying build/lib.linux-x86_64-3.5/torch/version.py -> /usr/local/lib/python3.5/dist-packages/torch
copying build/lib.linux-x86_64-3.5/caffe2/proto/caffe2_legacy_pb2.py -> /usr/local/lib/python3.5/dist-packages/caffe2/proto
copying build/lib.linux-x86_64-3.5/caffe2/proto/metanet_pb2.py -> /usr/local/lib/python3.5/dist-packages/caffe2/proto
copying build/lib.linux-x86_64-3.5/caffe2/proto/prof_dag_pb2.py -> /usr/local/lib/python3.5/dist-packages/caffe2/proto
copying build/lib.linux-x86_64-3.5/caffe2/proto/torch_pb2.py -> /usr/local/lib/python3.5/dist-packages/caffe2/proto
copying build/lib.linux-x86_64-3.5/caffe2/proto/predictor_consts_pb2.py -> /usr/local/lib/python3.5/dist-packages/caffe2/proto
copying build/lib.linux-x86_64-3.5/caffe2/proto/caffe2_pb2.py -> /usr/local/lib/python3.5/dist-packages/caffe2/proto
copying build/lib.linux-x86_64-3.5/caffe2/proto/hsm_pb2.py -> /usr/local/lib/python3.5/dist-packages/caffe2/proto
byte-compiling /usr/local/lib/python3.5/dist-packages/torch/version.py to version.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/caffe2/proto/caffe2_legacy_pb2.py to caffe2_legacy_pb2.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/caffe2/proto/metanet_pb2.py to metanet_pb2.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/caffe2/proto/prof_dag_pb2.py to prof_dag_pb2.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/caffe2/proto/torch_pb2.py to torch_pb2.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/caffe2/proto/predictor_consts_pb2.py to predictor_consts_pb2.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/caffe2/proto/caffe2_pb2.py to caffe2_pb2.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/caffe2/proto/hsm_pb2.py to hsm_pb2.cpython-35.pyc
running install_egg_info
running egg_info
writing dependency_links to torch.egg-info/dependency_links.txt
writing top-level names to torch.egg-info/top_level.txt
writing entry points to torch.egg-info/entry_points.txt
writing torch.egg-info/PKG-INFO
reading manifest file 'torch.egg-info/SOURCES.txt'
writing manifest file 'torch.egg-info/SOURCES.txt'
removing '/usr/local/lib/python3.5/dist-packages/torch-1.1.0a0+98b333d-py3.5.egg-info' (and everything under it)
Copying torch.egg-info to /usr/local/lib/python3.5/dist-packages/torch-1.1.0a0+98b333d-py3.5.egg-info
running install_scripts
Installing convert-onnx-to-caffe2 script to /usr/local/bin
Installing convert-caffe2-to-onnx script to /usr/local/bin

[OSX] No module named 'caffe2.python.caffe2_pybind11_state'

MP4:pytorch yyoo$ export PYTHONPATH=`pwd`/build
MP4:pytorch yyoo$ python3 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
Failure
MP4:pytorch yyoo$ python3
Python 3.5.6 (default, Oct  3 2018, 19:57:36) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from caffe2.python import core
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
CRITICAL:root:Cannot load caffe2.python. Error: No module named 'caffe2.python.caffe2_pybind11_state'