Giter Site home page Giter Site logo

Comments (20)

rofirrim avatar rofirrim commented on August 24, 2024

Hi @Z10Frank,

could it be the absence of the whitespace? Looks like GCC may accept this without the whitespace.

Perhaps you can modify your build system to include it?

Kind regards,

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Thank you!
That's strange, compiling with gcc without Mercurium did not need the whitespace between the flag and the filename.
Anyway, adding the whitespace in the makefile the error message disappears and the *.d files for the dependencies are generated.
That's great progress, but the *.o files are still not generated, so something is still missing.

Maybe the compilation log can help?
compile_smilei_Mercurium.txt

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Hi @Z10Frank,

That's strange, compiling with gcc without Mercurium did not need the whitespace between the flag and the filename.

Yeah, probably GCC undertands this but Mercurium doesn't.

Maybe the compilation log can help? compile_smilei_Mercurium.txt

I don't see any obvious failure, do the commands printed in the log have actually executed?

Kind regards,

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Sorry, my bad, it was a dry print for make. Also, some environment variables were missing for the Python part, which is now causing some errors like:

Compiling src/Diagnostic/DiagnosticRadiationSpectrum.cpp
/gpfs/users/massimof/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:84:30: error: syntax error, unexpected __attribute__, expecting ',' or '}'
Compilation failed for file 'src/Collisions/CollisionalFusionDD.cpp'

Normally it does not do that with gcc, maybe Mercurium has an interaction with the Python part?

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024
Compiling src/Diagnostic/DiagnosticRadiationSpectrum.cpp
/gpfs/users/massimof/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:84:30: error: syntax error, unexpected __attribute__, expecting ',' or '}'
Compilation failed for file 'src/Collisions/CollisionalFusionDD.cpp'

That is probably Mercurium choking with C++ headers. The support of C++ in Mercurium has always been a bit difficult.

Do you need to use Mercurium (you only need it if you use Fortran or some of the unsupported features described in [1]) or you can use llvm-ompss instead?

Find it in https://github.com/bsc-pm/llvm/releases
Documentation: https://pm.bsc.es/ftp/ompss-2/doc/user-guide/llvm/index.html

Sorry for the detour with this.

[1] https://pm.bsc.es/ftp/ompss-2/doc/user-guide/llvm/index.html#unsupported-features

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Hello, thank you, sorry for my late answer, I managed to compile llvm-ompss 2 from the release page:

tar -xvf llvm-github-release-2021.11.tar
cd llvm-github-release-2021.11
cd llvm
mkdir install
cd install

module load llvm/9.0.1/gcc-9.2.0 
module load anaconda2/2019.10/gcc-9.2.0 
module load openmpi/4.0.2/gcc-9.2.0
module load cmake/3.16.2/intel-19.0.3.199 

export INSTALLATION_PREFIX_NANOS6=/gpfs/users/massimof/nanos6/install
export INSTALLATION_PREFIX=/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install

cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$INSTALLATION_PREFIX -DLLVM_ENABLE_PROJECTS=clang -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON -DCLANG_DEFAULT_NANOS6_HOME=$INSTALLATION_PREFIX_NANOS6 

make 
make install

Now, compiling the code with llvm (adding the flag --ompss-2 in the makefile and using SMILEICXX ?= mpic++):

module load openmpi/4.0.2/gcc-9.2.0 
module load hdf5/1.10.7/gcc-9.2.0-openmpi

export HDF5_ROOT_DIR=/gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/hdf5-1.10.7-j6enx3ruksp5mjo5ed6lhywrzozihamq

export I_MPI_CXX=/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/bin/clang++ 
export OMPI_CXX=/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/bin/clang++

export LD_LIBRARY_PATH=/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/lib:$LD_LIBRARY_PATH

make -j 10

an error occurs:

/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/bin/clang++: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory

In general I can't find the library libc++.so.1 in the installation folder of llvm, so even adding the lib subfolder to the LD_LIBRARY_PATH does not find it. Was there an error in my build of llvm from the last release present in https://github.com/bsc-pm/llvm/releases?

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Hi @Z10Frank

In general I can't find the library libc++.so.1 in the installation folder of llvm, so even adding the lib subfolder to the LD_LIBRARY_PATH does not find it. Was there an error in my build of llvm from the last release present in https://github.com/bsc-pm/llvm/releases?

This is strange. I am not sure why are you seeing this.

What is happening is that the clang++ binary that you have compiled is using the LLVM C++ library (libc++.so). When compiling with g++ the GNU C++ library is used (libstdcxx.so). So it looks like you have compiled LLVM with clang, which is OK (AFAICT g++ can only use libstdcxx.so).

Perhaps the clang you're using to compile llvm-ompss has been configured to use libc++ by default (I think this is a possibility) but somehow the library is not found in your system.

However, I would expect you to be able to find those libraries in general when running a program compiled with the clang you have used to compile llvm-ompss. Otherwise a very simple C++ "hello world" should fail for you as well.

So to be fair I don't have a good answer to your problem. Can you look into your environment and verify that the compiler you used to compile llvm-ompss can be used to compile a C++ "hello world" that you can actually run?

Kind regards,

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Thank you, indeed the Hello_World.cpp test with my environment creates the error:

/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/bin/clang++: error while loading shared libraries: libtinfow.so.6: cannot open shared object file: No such file or directory

Should I try to build llvm-ompss with gcc instead?

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Hi @Z10Frank,

This is confusing. We're using one compiler to compile the llvm-ompss compiler. Forget about the latter (llvm-ompss compiler), you care about the first one.

You can check about the compiler used to compile llvm-ompss, opening the CMakeCache.txt that you will find in the directory where you ran cmake and there should be a CMAKE_CXX_COMPILER option. This is the compiler you used to compile.

Can you try to use that compiler to compile a simple hello world?

// hello.cc
#include <iostream>
int main() { std::cout << "Hello C++\n"; }
$ <the-compiler-you-found-in-cmakecache> -o hello hello.cc

Now try to run hello

$ ./hello

Does it work for you?

Kind regards,

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Thank you, in the cluster I am working on there is an installed version of LLVM (not llvm-ompss), which works perfectly when compiling Smilei, with this environment

module load anaconda2/2019.10/gcc-9.2.0
module load hdf5/1.10.7/gcc-9.2.0-openmpi
module load llvm/9.0.1/gcc-9.2.0
module load openmpi/4.0.2/gcc-9.2.0
export LANG=en_US.utf8
export LC_ALL=en_US.utf8

export I_MPI_CXX=clang++
export HDF5_ROOT_DIR=/gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/hdf5-1.10.7-j6enx3ruksp5mjo5ed6lhywrzozihamq

In the associated directory, /gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/llvm-9.0.1-t4ctj2mm5onq5s2qrug5pvnzdx6tsfut/lib (which is automatically appended to the LD_LIBRARY_PATH when loading the module) there is for example the file libc++abi.so.1 , which is absent from my build of llvm-ompss. A simple test for hello world in c++ and even a more complex one like compiling Smilei works perfectly with this installation present in the machine, so I suspect that something is misplaced in how I built llvm-ompss.

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Thank you, in the cluster I am working on there is an installed version of LLVM (not llvm-ompss), which works perfectly when compiling Smilei, with this environment

module load anaconda2/2019.10/gcc-9.2.0
module load hdf5/1.10.7/gcc-9.2.0-openmpi
module load llvm/9.0.1/gcc-9.2.0
module load openmpi/4.0.2/gcc-9.2.0
export LANG=en_US.utf8
export LC_ALL=en_US.utf8

In the associated directory, /gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/llvm-9.0.1-t4ctj2mm5onq5s2qrug5pvnzdx6tsfut/lib (which is automatically appended to the LD_LIBRARY_PATH when loading the module) there is for example the file libc++abi.so.1 , which is absent from my build of llvm-ompss.

Ok but this is not the problem you're having. The problem is that you cannot execute the compiler that you just have compiled. And you can't because the libc++.so library is not found. And you should observe a similar problem if you compile a hello world with the compiler of your environment (not the compiler that you are compiler).

Can you verify that your clang compiler is operating as expected for a hello world such as the one I showed you above?

If it does work, ldd hello on the binary will tell you where libc++.so is found. And then run ldd on the clang++ of llvm-ompss that is failing for you and let's try to investigate from here.

Kind regards,

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Ok, thank you, so with the clang++ compiler of the machine that works (ldd hello, where hello is compiled with clang++ -o hello Hello_world.cpp ):

linux-vdso.so.1 =>  (0x00007ffdd6d72000)
	libc++.so.1 => /gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/llvm-9.0.1-t4ctj2mm5onq5s2qrug5pvnzdx6tsfut/lib/libc++.so.1 (0x00002b999b1bb000)
	libc++abi.so.1 => /gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/llvm-9.0.1-t4ctj2mm5onq5s2qrug5pvnzdx6tsfut/lib/libc++abi.so.1 (0x00002b999b298000)
	libm.so.6 => /lib64/libm.so.6 (0x00002b999b3bc000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b999b6be000)
	libc.so.6 => /lib64/libc.so.6 (0x00002b999b8d4000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b999bca2000)
	librt.so.1 => /lib64/librt.so.1 (0x00002b999bebe000)
	/lib64/ld-linux-x86-64.so.2 (0x00002b999b198000)

I don't know if it is important, but the llvm on the machine was compiled with gcc-9.2.0.

Now, using ldd on the clang++ created from my build of llvm-ompss (ldd /gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/bin/clang++) I obtain:

linux-vdso.so.1 =>  (0x00007ffce5901000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00002abd81306000)
	librt.so.1 => /lib64/librt.so.1 (0x00002abd81522000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00002abd8172a000)
	libm.so.6 => /lib64/libm.so.6 (0x00002abd8192e000)
	libz.so.1 => /lib64/libz.so.1 (0x00002abd81c30000)
	libtinfow.so.6 => not found
	libc++.so.1 => /gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/llvm-9.0.1-t4ctj2mm5onq5s2qrug5pvnzdx6tsfut/lib/libc++.so.1 (0x00002abd81116000)
	libc++abi.so.1 => /gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/llvm-9.0.1-t4ctj2mm5onq5s2qrug5pvnzdx6tsfut/lib/libc++abi.so.1 (0x00002abd811f3000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002abd81e46000)
	libc.so.6 => /lib64/libc.so.6 (0x00002abd8205c000)
	/lib64/ld-linux-x86-64.so.2 (0x00002abd810e2000)

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Ok thanks!

libtinfow.so.6 => not found

this seems the culprit. Looks like you will need ncurses? Perhaps you need to load a module in your environment.

Kind regards,

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Thank you, I have installed ncurses, now the command ldd /gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/bin/clang++ seems to show everything in place:

linux-vdso.so.1 =>  (0x00007ffd5212f000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b47fff95000)
	librt.so.1 => /lib64/librt.so.1 (0x00002b48001b1000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00002b48003b9000)
	libz.so.1 => /gpfs/softs/spack/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/zlib-1.2.9-jthi6nubs2nrhi47wv7yexyau5v27cwu/lib/libz.so.1 (0x00002b48005bd000)
	libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00002b48007d4000)
	libstdc++.so.6 => /gpfs/softs/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-11.2.0-mpv3i3uebzvnvz7wxn6ywysd5hftycj3/lib64/libstdc++.so.6 (0x00002b48009fe000)
	libm.so.6 => /lib64/libm.so.6 (0x00002b4800e05000)
	libgcc_s.so.1 => /gpfs/softs/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-11.2.0-mpv3i3uebzvnvz7wxn6ywysd5hftycj3/lib64/libgcc_s.so.1 (0x00002b4801107000)
	libc.so.6 => /lib64/libc.so.6 (0x00002b4801320000)
	/lib64/ld-linux-x86-64.so.2 (0x00002b47ffd71000)

However when building Smilei this error occurs: fatal error: 'omp.h' file not found (I know that OpenMP will not be used when OmpSs-2 is activated, but the headers are still included). To avoid putting preprocessor directives before the include of 'omp.h', I wanted to build also OpenMP with the llvm-ompSs together with clang++, so restarting the process:

cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$INSTALLATION_PREFIX -DLLVM_ENABLE_PROJECTS="clang;openmp" -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON -DCLANG_DEFAULT_NANOS6_HOME=$INSTALLATION_PREFIX_NANOS6

The make command afterwards goes well, but the make install command gives this error:

CMake Error at projects/openmp/runtime/src/cmake_install.cmake:47 (file):
  file INSTALL cannot find
  "/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/lib/libomp.so":
  No such file or directory.
Call Stack (most recent call first):
  projects/openmp/runtime/cmake_install.cmake:42 (include)
  projects/openmp/cmake_install.cmake:42 (include)
  projects/cmake_install.cmake:42 (include)
  cmake_install.cmake:57 (include)


make: *** [install] Error 1

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Hi @Z10Frank

I'm at loss here. I've never seen this error before and I'd expect your cmake line to work fine when building openmp.

CMake Error at projects/openmp/runtime/src/cmake_install.cmake:47 (file):
  file INSTALL cannot find
  "/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install/lib/libomp.so":
  No such file or directory.
Call Stack (most recent call first):
  projects/openmp/runtime/cmake_install.cmake:42 (include)
  projects/openmp/cmake_install.cmake:42 (include)
  projects/cmake_install.cmake:42 (include)
  cmake_install.cmake:57 (include)


make: *** [install] Error 1

Kind regards,

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Hello, thank you, is there any intermediate step missing?
I mean, after downloading the llvm-ompss version, in theory can I directly compile clang and OpenMP like this?

tar -xvf llvm-github-release-2021.11.tar
cd llvm-github-release-2021.11
cd llvm
mkdir install
cd install

module load gcc/11.2.0/gcc-4.8.5 
module load anaconda2/2019.10/gcc-9.2.0 
module load cmake/3.16.2/intel-19.0.3.199 
module load openmpi/4.0.2/gcc-9.2.0
module load zlib/1.2.9/gcc-9.2.0

export INSTALLATION_PREFIX_NANOS6=/gpfs/users/massimof/nanos6/install
export INSTALLATION_PREFIX=/gpfs/workdir/massimof/llvm-github-release-2021.11/llvm/install

cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$INSTALLATION_PREFIX -DLLVM_ENABLE_PROJECTS="clang;openmp" -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON -DCLANG_DEFAULT_NANOS6_HOME=$INSTALLATION_PREFIX_NANOS6

make -j 10
make install

Specifically, is there something to build with LLVM before, to enable the subsequent LLVM compilation of clang and OpenMP? It is my first experience with LLVM, so any crucial step you may consider obvious is probably missing from my attempts.

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Hi @Z10Frank,

I tried to reproduce your problem without success.

I used the following steps in case you want to try.

$ wget https://github.com/bsc-pm/llvm/archive/refs/tags/github-release-2021.11.tar.gz
$ tar xf github-release-2021.11.tar.gz 
$ cd llvm-github-release-2021.11/
$ mkdir build
$ cd build/
$ make ../llvm -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX -DLLVM_ENABLE_PROJECTS="clang;openmp"  -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON -DCLANG_DEFAULT_NANOS6_HOME=$NANOS6_INSTALL
$ make -j$(nproc)
$ make install

My environment is an Ubuntu 18.04 LTS and cmake is 3.19.3.

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Thank you! Now I hope we are converging, the build of clang and OpenMP seems to arrive almost to the end. My error was to create a destination install directory that was also the place where CMake was called to build clang and OpenMP.

Creating a build directory, different from the $INSTALL_PREFIX directory like in your commands, made all smooth except for the make install step:

Install the project...
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:41 (file):
  file cannot create directory: /lib/cmake/llvm.  Maybe need administrative
  privileges.

Since I have no administrative privileges in this cluster, is it possible to specify a different path for CMake to create the llvm directory in this last step?

from mcxx.

rofirrim avatar rofirrim commented on August 24, 2024

Since I have no administrative privileges in this cluster, is it possible to specify a different path for CMake to create the llvm directory in this last step?

I didn't need administrative privileges earlier. cmake should honour CMAKE_INSTALL_PREFIX here.

Maybe something is still off in your environment?

from mcxx.

Z10Frank avatar Z10Frank commented on August 24, 2024

Thank you! Fixing the environment the LLVM-OmpSs compiler with OpenMP is built successfully, now I'll try to use it to compile Smilei.

from mcxx.

Related Issues (17)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.