Make unit tests work with and without OpenMP

[llnl]

Make unit tests work with and without OpenMP

[davidbeckingsale]

@DavidPoliakoff - this looks fixed here. I'm going to close the issue.

[jeffhammond]

Update: I am stupid 😃 I should have been using -DENABLE_OPENMP=Off instead of -DRAJA_ENABLE_OPENMP=Off. It might be useful to document the correct CMake variables in the README...

Maybe I am stupid, but CMake fails when it does not find OpenMP when I do not specify whether I want it or not (see below). Is there no way to allow CMake to use OpenMP only if available? I do not want to explicitly disable it for Clang, although that seems to the be the only workaround for now.

cmake .. -DCMAKE_CXX_COMPILER=clang++-3.9 -DCMAKE_C_COMPILER=clang-3.9 -DCMAKE_INSTALL_PREFIX=/home/travis/build/ParRes/Kernels/PRK-deps/raja -DRAJA_ENABLE_TBB=On
-- The CXX compiler identification is Clang 3.9.1
-- The C compiler identification is Clang 3.9.0
-- Check for working CXX compiler: /usr/bin/clang++-3.9
-- Check for working CXX compiler: /usr/bin/clang++-3.9 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/local/clang-3.9.0/bin/clang-3.9
-- Check for working C compiler: /usr/local/clang-3.9.0/bin/clang-3.9 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/bin/git (found version "2.14.1") 
-- Git Executable: /usr/bin/git
-- Git Version: 2.14.1
-- MPI Support is OFF
-- CUDA Support is Off
-- Failed to locate valgrind executable (missing:  VALGRIND_EXECUTABLE) 
-- OpenMP Support is On
-- Try OpenMP C flag = [ ]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [ ]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [/openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-Qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-xopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [+Oopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-qsmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-mp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
CMake Error at /home/travis/build/ParRes/Kernels/PRK-deps/cmake/share/cmake-3.4/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
  Could NOT find OpenMP (missing: OpenMP_CXX_FLAGS)
Call Stack (most recent call first):
  /home/travis/build/ParRes/Kernels/PRK-deps/cmake/share/cmake-3.4/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
  /home/travis/build/ParRes/Kernels/PRK-deps/cmake/share/cmake-3.4/Modules/FindOpenMP.cmake:231 (find_package_handle_standard_args)
  blt/cmake/SetupCompilerOptions.cmake:53 (find_package)
  blt/SetupBLT.cmake:176 (include)
  CMakeLists.txt:58 (include)
-- Configuring incomplete, errors occurred!
See also "/home/travis/build/ParRes/Kernels/RAJA/build/CMakeFiles/CMakeOutput.log".
See also "/home/travis/build/ParRes/Kernels/RAJA/build/CMakeFiles/CMakeError.log".

https://travis-ci.org/ParRes/Kernels/jobs/298165865 has details.

[davidbeckingsale]

Hey @jeffhammond, we currently have OpenMP on by default, and the configuration should fail if the feature is not found. The alternatives would be to either turn it off by default, or turn it off if it's not found. We are trying to avoid the latter option as it means build options can change without the user realizing.

[jeffhammond]

@davidbeckingsale Does CMake allow the user to specify -DENABLE_OPENMP=Auto, where Auto means "use it if available, otherwise not" i.e. "do not error if OpenMP is missing"? That is what I need here.

[davidbeckingsale]

It doesn't, the boolean options are On and Off. We used to have On function like auto, but it makes me a little uncomfortable doing that.

[jeffhammond]

What's the value in forcing the user to run CMake twice to determine that some versions of Clang don't support OpenMP? If On functions like Auto, it's not going to break user experience with any compiler that supports OpenMP.

It's not like I can't work around this, but it sort of defeats the purpose of having an intelligent build system if it isn't intelligent enough to support Clang's lack of universal OpenMP support without human intervention.

[davidbeckingsale]

I think the difference in thinking if between "What features do I want?" and "What features does my compiler support?". For your kernels, obviously enabling all supported features makes sense, but for an application, they are more likely to be building a specific set of features.

If there was an Auto option that was a CMake construct, that would be awesome and we would use it, since the semantic is clearly defined. My concern is what happens if we make On mean Auto, and a user requests OpenMP, it isn't found, but CMake passes and they build everything, but then when they try and use it in their application they get a bunch of errors.

[jeffhammond]

@davidbeckingsale If I was building applications on top of RAJA, I'd write them to be agnostic to the threading back-end, because I don't see a reason why one shouldn't fall back on TBB if OpenMP support is busted, and either permit sequential execution or error otherwise.

#if defined(RAJA_ENABLE_OPENMP)
typedef RAJA::NestedPolicy<RAJA::ExecList<RAJA::omp_parallel_for_exec, RAJA::seq_exec>> parallel_for;
#elif defined(RAJA_ENABLE_TBB)
typedef RAJA::NestedPolicy<RAJA::ExecList<RAJA::tbb_for_exec, RAJA::seq_exec>> parallel_for;
#elif defined(APP_SERIAL_OK)
#warning No thread-parallelism enabled!!!!!!!!!!!!!!!
typedef RAJA::NestedPolicy<RAJA::ExecList<RAJA::seq_exec, RAJA::seq_exec>> parallel_for;
#else
#error You wanted parallelism but didn't get it.
#endif

I like RAJA and would use it applications if I wrote those anymore, but I think it's rather silly that the expected usage of RAJA is deliberately back-end specific such that the code won't compile if a particular back-end is disabled/missing.