Quick compilation test on macOS Sierra 10.12.6 failing

[pytorch]

Quick compilation test on macOS Sierra 10.12.6 failing

[nadavrot]

Interesting. We had problems like this in the past. I just tried on Linux and Mac and I am not seeing this error.

Are you using Make or Ninja to build the code?

@compnerd Do you know what's going on?

[bwasti]

using cmake + make

[nadavrot]

ok. trying to build glow with 'make' to reproduce the problem.

[nadavrot]

I can reproduce this issue locally with Make but not with Ninja.

[nadavrot]

[ 16%] Completed 'llvm'
[ 16%] Built target llvm
Scanning dependencies of target llvm-install
[ 23%] Built target llvm-install
Scanning dependencies of target Support
[ 24%] Building CXX object src/glow/Support/CMakeFiles/Support.dir/Random.cpp.o
[ 25%] Building CXX object src/glow/Support/CMakeFiles/Support.dir/Support.cpp.o
[ 27%] Linking CXX static library libSupport.a
[ 27%] Built target Support
Scanning dependencies of target Base
[ 28%] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Train.cpp.o
[ 28%] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Tensor.cpp.o
[ 29%] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Type.cpp.o
[ 30%] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Image.cpp.o
[ 31%] Linking CXX static library libBase.a
[ 31%] Built target Base
Scanning dependencies of target NodeGen
[ 32%] Building CXX object tools/ClassGen/CMakeFiles/NodeGen.dir/NodeGen.cpp.o
In file included from /Users/nrotem/devel/glow/tools/ClassGen/NodeGen.cpp:2:
In file included from /Users/nrotem/devel/glow/tools/ClassGen/NodeBuilder.h:5:
/Users/nrotem/devel/glow/include/glow/Support/Support.h:4:10: fatal error: 'llvm/ADT/ArrayRef.h' file not found
#include "llvm/ADT/ArrayRef.h"
^~~~~~~~~~~~~~~~~~~~~
1 error generated.

[nadavrot]

@bwasti : a quick workaround would be to use Ninja instead of 'make'.

[nadavrot]

Ok, I know what the problem is:

Adding this to the NodeGen cmake file should fix the problem.

tools/ClassGen/CMakeLists.txt

+target_link_libraries(NodeGen

•                  PUBLIC
  

•                    Base
  

•                    Support)
  

[nadavrot]

target_link_libraries(NodeGen
PUBLIC
Base
Support)

[bwasti]

fails with ninja too 😢

bwasti-mbp:/Users/bwasti/Glow/build $ git diff
diff --git a/tools/ClassGen/CMakeLists.txt b/tools/ClassGen/CMakeLists.txt
index 835d410..9957bfb 100644
--- a/tools/ClassGen/CMakeLists.txt
+++ b/tools/ClassGen/CMakeLists.txt
@@ -7,6 +7,11 @@ add_executable(InstrGen
                  InstrGen.cpp
                  InstrBuilder.cpp)

+target_link_libraries(NodeGen
+    PUBLIC
+    Base
+    Support)
+
 target_link_libraries(InstrGen
                       PUBLIC
                         Base

errors:

[7/66] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Tensor.cpp.o
FAILED: src/glow/Base/CMakeFiles/Base.dir/Tensor.cpp.o
/Applications/Xcode_9.0_fb.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++   -DWITH_PNG=1 -Iinclude -Iinclude -I. -I/Library/Frameworks/Mono.framework/Headers -isystem external/llvm-prefix/include -Wall -fno-exceptions -fno-rtti -g -fno-omit-frame-pointer -O0   -std=c++14 -MD -MT src/glow/Base/CMakeFiles/Base.dir/Tensor.cpp.o -MF src/glow/Base/CMakeFiles/Base.dir/Tensor.cpp.o.d -o src/glow/Base/CMakeFiles/Base.dir/Tensor.cpp.o -c /Users/bwasti/glow/src/glow/Base/Tensor.cpp
In file included from /Users/bwasti/glow/src/glow/Base/Tensor.cpp:3:
include/glow/Base/Tensor.h:158:26: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    auto dim = t->dims().drop_front();
               ~~~~~~~~~ ^
include/glow/Base/Tensor.h:173:35: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    auto onceSliceDim = t->dims().drop_front();
                        ~~~~~~~~~ ^
include/glow/Base/Tensor.h:175:35: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    assert(onceSliceDim == dims().drop_front() && "Invalid slice size");
                           ~~~~~~ ^
/usr/include/assert.h:93:25: note: expanded from macro 'assert'
    (__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
                        ^
In file included from /Users/bwasti/glow/src/glow/Base/Tensor.cpp:3:
include/glow/Base/Tensor.h:357:32: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    Tensor slice(elemTy, sizes.drop_front());
                         ~~~~~ ^
4 errors generated.
[8/66] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Train.cpp.o
FAILED: src/glow/Base/CMakeFiles/Base.dir/Train.cpp.o
/Applications/Xcode_9.0_fb.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++   -DWITH_PNG=1 -Iinclude -Iinclude -I. -I/Library/Frameworks/Mono.framework/Headers -isystem external/llvm-prefix/include -Wall -fno-exceptions -fno-rtti -g -fno-omit-frame-pointer -O0   -std=c++14 -MD -MT src/glow/Base/CMakeFiles/Base.dir/Train.cpp.o -MF src/glow/Base/CMakeFiles/Base.dir/Train.cpp.o.d -o src/glow/Base/CMakeFiles/Base.dir/Train.cpp.o -c /Users/bwasti/glow/src/glow/Base/Train.cpp
In file included from /Users/bwasti/glow/src/glow/Base/Train.cpp:3:
In file included from include/glow/Base/Train.h:4:
include/glow/Base/Tensor.h:158:26: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    auto dim = t->dims().drop_front();
               ~~~~~~~~~ ^
include/glow/Base/Tensor.h:173:35: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    auto onceSliceDim = t->dims().drop_front();
                        ~~~~~~~~~ ^
include/glow/Base/Tensor.h:175:35: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    assert(onceSliceDim == dims().drop_front() && "Invalid slice size");
                           ~~~~~~ ^
/usr/include/assert.h:93:25: note: expanded from macro 'assert'
    (__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
                        ^
In file included from /Users/bwasti/glow/src/glow/Base/Train.cpp:3:
In file included from include/glow/Base/Train.h:4:
include/glow/Base/Tensor.h:357:32: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    Tensor slice(elemTy, sizes.drop_front());
                         ~~~~~ ^
4 errors generated.
[9/66] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Image.cpp.o
FAILED: src/glow/Base/CMakeFiles/Base.dir/Image.cpp.o
/Applications/Xcode_9.0_fb.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++   -DWITH_PNG=1 -Iinclude -Iinclude -I. -I/Library/Frameworks/Mono.framework/Headers -isystem external/llvm-prefix/include -Wall -fno-exceptions -fno-rtti -g -fno-omit-frame-pointer -O0   -std=c++14 -MD -MT src/glow/Base/CMakeFiles/Base.dir/Image.cpp.o -MF src/glow/Base/CMakeFiles/Base.dir/Image.cpp.o.d -o src/glow/Base/CMakeFiles/Base.dir/Image.cpp.o -c /Users/bwasti/glow/src/glow/Base/Image.cpp
In file included from /Users/bwasti/glow/src/glow/Base/Image.cpp:3:
In file included from include/glow/Base/Image.h:4:
include/glow/Base/Tensor.h:158:26: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    auto dim = t->dims().drop_front();
               ~~~~~~~~~ ^
include/glow/Base/Tensor.h:173:35: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    auto onceSliceDim = t->dims().drop_front();
                        ~~~~~~~~~ ^
include/glow/Base/Tensor.h:175:35: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    assert(onceSliceDim == dims().drop_front() && "Invalid slice size");
                           ~~~~~~ ^
/usr/include/assert.h:93:25: note: expanded from macro 'assert'
    (__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
                        ^
In file included from /Users/bwasti/glow/src/glow/Base/Image.cpp:3:
In file included from include/glow/Base/Image.h:4:
include/glow/Base/Tensor.h:357:32: error: no member named 'drop_front' in 'llvm::ArrayRef<unsigned long>'
    Tensor slice(elemTy, sizes.drop_front());
                         ~~~~~ ^
4 errors generated.
[10/66] Building CXX object src/glow/Base/CMakeFiles/Base.dir/Type.cpp.o
ninja: build stopped: subcommand failed.

[nadavrot]

I was just able to complete a fresh build using 'make'. I am not sure what's going on. What version of CMake are you using?

[bwasti]

3.7.2 from homebrew I believe

bwasti-mbp:/Users/bwasti/Glow $ cmake --version
cmake version 3.7.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).

[bwasti]

some extra data points:

1. CMake warned about my protobuf compiler not matching the runtime library (just fixed that and trying to recompile), but I doubt this is the issue

2. I'm using cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DENABLE_TESTING=YES .. rather than ../glow as in the instructions on the readme (I assumed those are old/typo)

3. I'm using cmake --build instead of some ninja command

[nadavrot]

Sorry for the delay in response. We are on a family vacation.

I build Glow from outside the source tree. I actually have two build directories: build_Debug and build_Release. Debug builds are really slow because we have assertions all over the place, but are excellent for debugging things. And Release builds are important for testing the large programs, like RESNET50.

We should clean up the docs and remove -DENABLE_TESTING because it is always on.

So I build the compiler like this:
cd build_Debug; cmake -G Ninja ../glow
cd build_Release; cmake -G Ninja ../glow -DCMAKE_BUILD_TYPE=Release

[bwasti]

I can't seem to figure out why drop_front() is not being picked up for ArrayRef<unsigned long>, nor can I determine why #175 fixes it, but I thought it'd be a good data point for figuring out this issue (no need to merge the change, as it is less readable imo)