OpenEXR dwaLookups gets Segmentation fault: 11 [OSX Catalina - 10.15]

[mitsuba-renderer]

OpenEXR dwaLookups gets Segmentation fault: 11 [OSX Catalina - 10.15]

[guy4261]

In the CMAKE_CXX_FLAGS for Clang?

# Line 13 of ext/openexr/CMakeLists.txt
-  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-switch -Wno-tautological-compare -Wno-deprecated-register")
+  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-switch -Wno-tautological-compare -Wno-deprecated-register -fno-stack-check")

[wjakob]

Then lldb -- mitsuba sph.xml, run, and bt.

[nmatsuda]

In the CMAKE_CXX_FLAGS for Clang?

# Line 13 of ext/openexr/CMakeLists.txt
-  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-switch -Wno-tautological-compare -Wno-deprecated-register")
+  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-switch -Wno-tautological-compare -Wno-deprecated-register -fno-stack-check")

yes - that's where I put it

[guy4261]

Works just as well on a Release build (i.e. without -DCMAKE_BUILD_TYPE=Debug, where Release gets picked as the default).

I'll conclude that upgrading XCode via the App Store and reaching a state where

$ xcodebuild -version
Xcode 11.3.1
Build version 11C504

seems to have solved it. I also had to run XCode (the GUI app) and let it install some "tools" (it prompted be on the first run after the upgrade and I allowed it. Don't remember the exact wording of that prompted message box). After XCode installed its tools, I could see my clang was upgraded as well (compare with the version in which the build failed):

$ clang --version
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin19.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

No additional code changes (such as adding -fno-stack-check) were required.

Closing this as solved. Thanks!!!

[leroyvn]

Sorry to bring this up again: I got the same issue upon upgrading to XCode 11.4. I rolled back to 11.3.1 and it works fine again.

[arpit15]

I confirm that Xcode 11.4 upgrade works with the proposed addition of fno-stack-check flag

[wjakob]

That's very strange. What kind of processor architecture / Linux version is this running on? What compiler version did you use?

[guy4261]

Hi @wjakob - that's on OSX, not Linux. My clang[++] versions are Apple clang version 11.0.0 (clang-1100.0.33.12).

$ clang --version  # output for `clang++ --version` is identical
Apple clang version 11.0.0 (clang-1100.0.33.12)
Target: x86_64-apple-darwin19.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[andrewwillmott]

In case it's useful as a sanity check, I've successfully built and run scenes on Mojave (10.14.3) with an older version of clang (clang-1000.11.45.5).

Running ext_build/openexr/OpenEXR/IlmImf/./dwaLookups directly completes successfully.

Seems like it's an OpenEXR issue. Maybe try downloading that and compiling directly as a cross check?

[wjakob]

Super weird. Can you build in debug mode & attach LLDB to the crashing dwaLookups? Maybe a backtrace will shed light on what's going on here.

[nmatsuda]

I just had this issue, LLDB throws "libdyld.dylib`stack_not_16_byte_aligned_error", which led me to this post:

Homebrew/homebrew-core#44776 (comment)

Apparently it's xcode 11 specific. Passing -fno-stack-check appears to resolve the issue.

[wjakob]

Amazing, thank you for the detective work! Does this only happen when building in debug mode, or also release mode? It would be a piece of cake to just always provide this parameter to the recursive OpenEXR build.

[nmatsuda]

It happens in both as far as I can tell. I put the parameter directly into ext/openexr/CMakeLists.txt, but that homebrew thread makes it sounds like passing it by default globally should be ok.

[guy4261]

@nmatsuda sounds good! So if compilation is done by

$ mkdir build
$ cd build
$ cmake -GNinja ..
$ ninja

where does the -fno-stack-check?

$ ninja  -fno-stack-check  # here?
$ cmake -GNinja -fno-stack-check  # or here?

Or otherwise - where do I add it in ext/openexr/CMakeLists.txt?

Thanks!

[guy4261]

The build now passes and mitsuba can be executed (after source setpath.sh). Sadly, the exr stage seems to cause trouble. Here's what I get when I try to render a sphere (XML taken from here):

# Taken from https://mitsuba2.readthedocs.io/en/latest/src/getting_started/file_format/#scene-file-format
$ cat sph.xml
<scene version="2.0.0">
    <shape type="sphere">
        <float name="radius" value="10"/>
    </shape>
</scene>

$ mitsuba sph.xml
2020-03-12 01:04:14 INFO  main  [mitsuba.cpp:194] Mitsuba version 2.0.0 (master[d30def81], Mac OS, 64bit, 8 threads, 8-wide SIMD)
2020-03-12 01:04:14 INFO  main  [mitsuba.cpp:195] Copyright 2019, Realistic Graphics Lab, EPFL
2020-03-12 01:04:14 INFO  main  [mitsuba.cpp:196] Enabled processor features: avx2 avx fma f16c sse4.2 x86_64
2020-03-12 01:04:14 INFO  main  [mitsuba.cpp:197] Using mode "packet_rgb"
2020-03-12 01:04:14 INFO  main  [xml.cpp:1117] Loading XML file "sph.xml" ..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/sphere.dylib" ..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/diffuse.dylib" ..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/uniform.dylib" ..
2020-03-12 01:04:14 WARN  main  [Scene] No sensors found! Instantiating a perspective camera..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/perspective.dylib" ..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/hdrfilm.dylib" ..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/gaussian.dylib" ..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/independent.dylib" ..
2020-03-12 01:04:14 WARN  main  [Scene] No integrator found! Instantiating a path tracer..
2020-03-12 01:04:14 INFO  main  [PluginManager] Loading plugin "plugins/path.dylib" ..
2020-03-12 01:04:14 INFO  main  [ShapeKDTree] Building a SAH kd-tree (1 primitives) ..
2020-03-12 01:04:14 INFO  main  [ShapeKDTree] Finished. (12 B of storage, took 0ms)
2020-03-12 01:04:14 INFO  main  [SamplingIntegrator] Starting render job (768x576, 4 samples, 8 threads)
Rendering [===============================================================================================] (82ms, ETA: 0ms)
2020-03-12 01:04:14 INFO  main  [SamplingIntegrator] Rendering finished. (took 82ms)
2020-03-12 01:04:14 INFO  main  [HDRFilm] ✔  Developing "sph.exr" ..
Segmentation fault: 11

[wjakob]

Interesting that OpenEXR is causing so much trouble (working perfectly fine here on MacOS / Mojave). Could you make a debug build, attach via LLDB, and get a backtrace?

[guy4261]

I'm on 10.15 (Catalina not Mojave).

Sure, how? Do I rm -rf * inside build, then this?

$ cmake -DCMAKE_BUILD_TYPE=Debug -GNinja ..
$ ninja

And afterwards lldb mitsuba sph.xml ? Or do I need more flags for lldb ?

[guy4261]

BTW since xcode has been mentioned as a culprit by @nmatsuda , I'm with 11.2.1:

$ xcodebuild -version
Xcode 11.2.1
Build version 11B500

This is not the latest - I can upgrade to 11.3.1 from the App Store.

[wjakob]

You can easily toggle the build mode using a tool like "ccmake" or "cmake-gui", then simply run "ninja" once more.

[guy4261]

Thanks for the lldb instructions! I thought the cmake part is as easy as this. I'll try with the cmake tools you mentioned.

[nmatsuda]

BTW since xcode has been mentioned as a culprit by @nmatsuda , I'm with 11.2.1:

$ xcodebuild -version
Xcode 11.2.1
Build version 11B500

This is not the latest - I can upgrade to 11.3.1 from the App Store.

I'm on 11.3.1 and unfortunately the issue is still present - I think 11.0 through 11.3 all use the same version of clang. But no issues so far after adding the compiler flag (I think the segfault you got at runtime might have been from one of your previous builds)

[guy4261]

OK now:

1. I upgraded XCode via the App store, now

$ xcodebuild -version
Xcode 11.3.1
Build version 11C504

2. I reset my git completely, including the -fno-stack-check change because I didn't want to skip a check in the build but then get a segfault during runtime.

3. I ran cmake -DCMAKE_BUILD_TYPE=Debug -GNinja .. && ninja.

Now (after source setpath.sh of course) - mitsuba sph.xml generates a proper sph.exr file!

Likewise cd resources/data/scenes && mitsuba hello.xml gives me the hello.exr with the famous teapot.

Thanks @nmatsuda and @wjakob !

I'll delete everything under build and will retry without the Debug flag. (@nmatsuda : I always do that, this is why I don't think the previous builds affected me as I always start with the build dir empty, correct me if I'm wrong about this please!).

[binorchen]

Hello @nmatsuda and @wjakob,
I got a similar issue during my compilation. Please help me out.
System information:

Ubuntu 16.04
cmake 3.13.4
ninja 1.5.1
clang-10
python 3.6.12
cuda-10.1
NVIDIA driver 418.87.00

Below is returned by cmake -GNinja .. -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc

-- The C compiler identification is Clang 10.0.1
-- The CXX compiler identification is Clang 10.0.1
-- Check for working C compiler: /usr/bin/clang-10
-- Check for working C compiler: /usr/bin/clang-10 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/clang++-10
-- Check for working CXX compiler: /usr/bin/clang++-10 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /home/user/anaconda3/envs/stylegan2/bin/python3.6 (found version "3.6.12")
-- Found PythonLibs: /home/user/anaconda3/envs/stylegan2/lib/libpython3.6m.so
-- Building the following variants of Mitsuba:
--  * scalar_rgb
--  * gpu_autodiff_rgb
-- Setting build type to 'Release' as none was specified.
-- Mitsuba: using libc++.
-- Mitsuba: building the Python plugin.
-- The CUDA compiler identification is NVIDIA 10.1.243
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Enoki: building the CUDA backend.
-- Enoki: building the autodiff backend.
-- Enoki: building the Python plugin.
-- pybind11 v2.4.dev4
-- LTO enabled
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
-- Found PNG: /usr/lib/x86_64-linux-gnu/libpng.so (found version "1.2.54")
-- Found JPEG: /usr/lib/x86_64-linux-gnu/libjpeg.so (found version "80")
-- TBB: using libc++.
-- Mitsuba: using builtin implementation for CPU ray tracing.
-- Mitsuba: using OptiX for GPU ray tracing.
-- Mitsuba: LTO support enabled.
-- Check if the system is big endian
-- Searching 16 bit integer
-- Using unsigned short
-- Check if the system is big endian - little endian
-- Mitsuba: sampling profiler enabled.
-- Found Sphinx: /home/user/anaconda3/envs/stylegan2/bin/sphinx-build
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/software/mitsuba2/build

And the error return by ninja:

[121/684] Building CXX object ext_build/openexr/OpenEXR/IlmImf/CMakeFiles/IlmImf-obj.dir/ImfConvert.cpp.o
../ext/openexr/OpenEXR/IlmImf/ImfConvert.cpp:110:31: warning: implicit conversion from 'unsigned int' to 'float' changes value from 4294967295 to 4294967296 [-Wimplicit-int-float-conversion]
    if (isInfinity (f) || f > UINT_MAX)
                            ~ ^~~~~~~~
/usr/lib/llvm-10/lib/clang/10.0.1/include/limits.h:56:37: note: expanded from macro 'UINT_MAX'
#define UINT_MAX  (__INT_MAX__  *2U +1U)
                   ~~~~~~~~~~~~~~~~~^~~
1 warning generated.
[200/684] Generating dwaLookups.h
FAILED: cd /home/user/software/mitsuba2/build/ext_build/openexr/OpenEXR/IlmImf && /home/user/software/mitsuba2/build/ext_build/openexr/OpenEXR/IlmImf/./dwaLookups > /home/user/software/mitsuba2/build/ext_build/openexr/OpenEXR/IlmImf/dwaLookups.h
/home/user/software/mitsuba2/build/ext_build/openexr/OpenEXR/IlmImf/./dwaLookups: symbol lookup error: /home/user/software/mitsuba2/build/ext_build/openexr/OpenEXR/IlmImf/./dwaLookups: undefined symbol: _ZTIN9IlmThread6ThreadE
[200/684] Building CXX object ext_build/enoki/CMakeFiles/enoki-python-cuda-autodiff.dir/src/python/cuda_autodiff_matrix.cpp.o
ninja: build stopped: subcommand failed.