Build Caffe2 on windows failed

[facebookarchive]

Build Caffe2 on windows failed

[yiwsun]

Add caffe2/build path to your PYTHONPATH. It was mentioned in the install tutorial for Ubuntu. Perhaps worth mentioning in the windows version too.

[ezineo]

Do you have ran
git clone --recursive https://github.com/caffe2/caffe2.git ?
And check out your C:/Users/sqlpythonadmin/Source/Repos/caffe2/third_party/protobuf path, where protobuf source code exists.

[yiwsun]

yes, I did, by using Git Plugin in Visual Studio with "recursive clone submodules" checked.
Is there a way to remove all existing caffe2 stuff and restart the process again? Would remove C:/Users/sqlpythonadmin/Source/Repos/caffe2 folder be enough?

[yiwsun]

After remove caffe2 folder, and another folder where protobuf was installed using conda install, rerun build_windows.bat, the issue with protobuf seems gone. But came another issue:
C:\Users\sqlpythonadmin\Source\Repos\caffe2\python\pybind_state.cc(194): error C3861: 'PyString_AsString': identifier
not found [C:\Users\sqlpythonadmin\Source\caffe2\build\caffe2\caffe2_pybind11_state.vcxproj]
C:\Users\sqlpythonadmin\Source\Repos\caffe2\python\pybind_state.cc(196): error C3861: 'PyString_AsString': identifier
not found [C:\Users\sqlpythonadmin\Source\caffe2\build\caffe2\caffe2_pybind11_state.vcxproj]
C:\Users\sqlpythonadmin\Source\Repos\caffe2\python\pybind_state.cc(314): error C3861: 'PyString_Check': identifier not
found [C:\Users\sqlpythonadmin\Source\caffe2\build\caffe2\caffe2_pybind11_state.vcxproj]
C:\Users\sqlpythonadmin\Source\Repos\caffe2\python\pybind_state.cc(819): error C3861: 'PyString_Check': identifier not
found [C:\Users\sqlpythonadmin\Source\caffe2\build\caffe2\caffe2_pybind11_state.vcxproj]
pybind_state_mkl.cc
Generating Code...
Done Building Project "C:\Users\sqlpythonadmin\Source\caffe2\build\caffe2\caffe2_pybind11_state.vcxproj" (default targe
ts) -- FAILED.

A quick search indicates "PyString_AsString" is defined in Python2, but I have installed Python3.5.2. Does it mean Caffe2 only works with Python2? Thanks.

[ezineo]

Yes, you get it.
Caffe2 doesn't support Python3 yet. You could use Anaconda2 instead.

[yiwsun]

Thank you ezineo! I managed to replace some deprecated Python2 methods with Python3 ones. and the caffe2 build pass. But when trying to import Caffe2 in Python:
from caffe2.python import core, workspace
from caffe2.proto import caffe2_pb2
it reported an error "ImportError: No module named 'caffe2'". Some discussion mentioned about pycaffe, but I was not able to find pycaffe in the build. Did I miss installing something?

[Yangqing]

Could you recursively list the files under the build folder, and especially where all the .so files are located?

[yiwsun]

There is no .so files under \build.
Attaching the output
Caffe2 build folder files-recursive listing.txt

[yiwsun]

Search results showed that even Caffe2.exe was not existing. So the "Build succeeded" message must be a false positive one.
Attaching the build output message. Could you help figure out what is missing? Is it due to I am using Python 3?
build_windows_bat_outpu-fixedProtobuft_fixedPyString_AsString.txt
Thanks!

[Yangqing]

Ah yeah, right now C2 does not support python3 yet, stay tuned - that is coming in the near future. For now switching to python 2.7 might be the easiest option.

[yiwsun]

Thank you Yangqing for the confirmation. I actually tried to install Caffe2 on another VM (windows server 2016 with Python2.7 (Anaconda2), VS2017Community). But it was not able to build caffe2.exe either, with CMakeError like:

• C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\CheckIncludeFile.c(1): fatal error C1083: Cannot open include file: 'pthread.h': No such file or directory [C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\cmTC_71456.vcxproj]
• C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\Debug\cmTC_01d7c.exe : fatal error LNK1120: 1 unresolved externals [C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\cmTC_01d7c.vcxproj]
• C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\src.cxx(4): error C3861: '__builtin_cpu_supports': identifier not found [C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\cmTC_a31f6.vcxproj]
• C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\src.cxx(1): fatal error C1083: Cannot open include file: 'glog/stl_logging.h': No such file or directory [C:\Users\pythonadmin\Source\Repos\caffe2\build\CMakeFiles\CMakeTmp\cmTC_22aa6.vcxproj]

Did you experience similar errors while testing on Windows? Any suggestions on how to resolve it?

[Yangqing]

Hmm, these all seem to be normal cmake check errors, other than the last line which seems to suggest that glog/stl_logging.h cannot be found. Are you building with glog on Windows? That is a path we have not verified before, so I wonder if you pass in -DUSE_GLOG=OFF things can pass.

[yiwsun]

Thanks for your suggestion Yangqing. Finally figured out, I missed an environment variable PYTHONPATH, after adding it, caffe2 works.

[radla001]

Can you please specify which PYTHONPATH have you added.... I have anaconda installed in "$USER\AppData\Local\Continuum\Anaconda2\python.exe" which is my PYTHONPATH in Windows 10 Desktop.

Also I can confirm that I have copied "caffe2_pybind11_state*.pyd" files to "$USER\AppData\Local\Continuum\Anaconda2\DLLs"

But no luck when I try to import caffe2 ">>from caffe2.python import core"
"ImportError: No Module named caffe2.python"

[radla001]

Thanks....it works for me now

[SWX-TJ]

I also have this question, when I try to import caffe2 ">>from caffe2.python import core"
"ImportError: No Module named caffe2.python",I have add caffe2/build PYTHONPATH in windows environment,but it didn't work,how can i do?