hfp/xconfigure

High-Performance configuration patterns and recipes.

XCONFIGURE

XCONFIGURE is a collection of configure wrapper scripts for various HPC applications. The purpose of the scripts is to configure the application in question to make use of Intel's software development tools (Intel Compiler, Intel MPI, Intel MKL). XCONFIGURE helps to rely on a "build recipe", which is known to expose the highest performance or to reliably complete the build process.

Contributions are very welcome!

Each application (or library) is hosted in a separate directory. To configure (and ultimately build) an application, one can rely on a single script which then downloads a specific wrapper into the current working directory (of the desired application).

wget --no-check-certificate https://github.com/hfp/xconfigure/raw/master/configure-get.sh
chmod +x configure-get.sh

echo "EXAMPLE: recipe for Quantum Espresso"
./configure-get.sh qe hsw

On systems without access to the Internet, one can download (or clone) the entire collection upfront. To configure an application, please open the config folder directly or use the documentation and then follow the build recipe of the desired application or library.

Documentation

• ReadtheDocs: online documentation with full text search: CP2K, ELPA, LIBINT, LIBXC, LIBXSMM, QE, and TF.
• PDF: a single documentation file.

Related Projects

• Spack Package Manager: http://computation.llnl.gov/projects/spack-hpc-package-manager
• EasyBuild / EasyConfig (University of Gent): https://github.com/easybuilders

Please note that XCONFIGURE has a narrower scope when compared to the above package managers.

xconfigure's Issues

Issue : compile cp2k-7.1 with libint-2

Hi
I used parallel studio 2020 to compile cp2k-7.1 with elpa, libint, libxc and libxsmm.
The compilations of libxsmm, libxc, elpa are fine,except libint.
After run make -j, there is no error but some warnings

ld: Warning: alignment 32 of symbol libint2_build_eri' in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o) is smaller than 64 in libint_f.o ld: Warning: alignment 32 of symbol libint2_build_eri1' in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o) is smaller than 64 in libint_f.o
ld: Warning: alignment 32 of symbol libint2_build_3eri' in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o) is smaller than 64 in libint_f.o ld: Warning: alignment 32 of symbol libint2_build_3eri1' in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o) is smaller than 64 in libint_f.o
ld: Warning: alignment 32 of symbol libint2_build_2eri' in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o) is smaller than 64 in libint_f.o ld: Warning: size of symbol libint2_build_2eri' changed from 392 in libint_f.o to 648 in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o)
ld: Warning: alignment 32 of symbol libint2_build_2eri1' in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o) is smaller than 64 in libint_f.o ld: Warning: size of symbol libint2_build_2eri1' changed from 288 in libint_f.o to 512 in /opt/cp2k/libint/lib/.libs/libint2.a(libint2_static_init.o)

When compiling cp2k, if I add LIBINTROOT, it will give some warnings

ld: Warning: alignment 32 of symbol libint2_build_3eri' in /opt/cp2k/libint/default-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/cp2k/cp2k/lib/Linux-x86-64-intelx/popt/libcp2kmain.a(libint_wrapper.o) ld: Warning: alignment 32 of symbol libint2_build_2eri' in /opt/cp2k/libint/default-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/cp2k/cp2k/lib/Linux-x86-64-intelx/popt/libcp2kmain.a(libint_wrapper.o)
ld: Warning: alignment 32 of symbol libint2_build_3eri' in /opt/cp2k/libint/default-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/cp2k/cp2k/lib/Linux-x86-64-intelx/popt/libcp2kmain.a(libint_wrapper.o) ld: Warning: alignment 32 of symbol libint2_build_2eri' in /opt/cp2k/libint/default-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/cp2k/cp2k/lib/Linux-x86-64-intelx/popt/libcp2kmain.a(libint_wrapper.o)

And if I ignore the warnings and continuously run cp2k
mpirun -n 4 cp2k.popt *.inp
the program crash, not give any information and not exit.
If not add libint , the program run perfectly.
How to solve this problem

Thanks

ld: warning: alignment 32 of symbol is smaller than 64 ...

ld: warning: alignment 32 of symbol libint2_build_3eri' in /opt/apps/libint/intel-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/apps/cp2k/lib/Linux-x86-64-intelx/psmp/libcp2kmain.a(libint_wrapper.o) ld: warning: alignment 32 of symbol libint2_build_2eri' in /opt/apps/libint/intel-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/apps/cp2k/lib/Linux-x86-64-intelx/psmp/libcp2kmain.a(libint_wrapper.o)
ld: warning: alignment 32 of symbol libint2_build_3eri' in /opt/apps/libint/intel-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/apps/cp2k/lib/Linux-x86-64-intelx/psmp/libcp2kmain.a(libint_wrapper.o) ld: warning: alignment 32 of symbol libint2_build_2eri' in /opt/apps/libint/intel-hsw/lib/libint2.a(libint2_static_init.o) is smaller than 64 in /opt/apps/cp2k/lib/Linux-x86-64-intelx/psmp/libcp2kmain.a(libint_wrapper.o)

Tried to compile cp2k v7.1.0 with intel compilers by following [https://github.com/hfp/xconfigure/tree/master/config/cp2k],
but I got the above warnings.

I have the E3-1231 v3 @ 3.40GHz installed on my server.
Therefore I switched from all configure-XXX-skx-YY.sh files to configure-XXX-hsw-YY.sh scripts.
And I dropped AVX in the make command.

The make test reports 1 FAILED, and 388 WRONG results.

The test errors look like the following:

relative error : 1.53237643e-02 > numerical tolerance = 1.0E-14
relative error : 2.10752261e-02 > numerical tolerance = 1.0E-14
relative error : 1.86119918e-01 > numerical tolerance = 1.0E-14
relative error : 5.14171374e-02 > numerical tolerance = 9e-14
relative error : 7.41726924e-02 > numerical tolerance = 1e-14
relative error : 1.08071772e-02 > numerical tolerance = 8.0E-13
relative error : 9.76937247e-03 > numerical tolerance = 1.0E-14
relative error : 2.28318315e-03 > numerical tolerance = 1e-04
relative error : 4.75900254e-03 > numerical tolerance = 2e-13
relative error : 9.80639405e-13 > numerical tolerance = 8e-13
Singularity> cat error_summary | grep 'relative error' | wc -l
388
Singularity>

Should I use all the skx scripts?

On another server with 8280L CPU,
use all skx scripts,
then make test reports only 1 FAILED.

In both cases, CentOS 8.2, intel 2020u1.

In both cases,
the failed output is this one,

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 0 PID 721729 RUNNING AT lab-itc
= KILLED BY SIGNAL: 11 (Segmentation fault)

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 1 PID 721730 RUNNING AT lab-itc
= KILLED BY SIGNAL: 9 (Killed)

"QS/regtest-rel/Hg_rel.inp.out" line 163 of 163 --100%-- col 1

I have checked these limit settings before invoke make test.

ulimit -d unlimited
ulimit -m unlimited
ulimit -s unlimited
ulimit -t unlimited
ulimit -v unlimited

Compiling QE 6.1 : configure-qe-hsw-omp.sh looks for a make.sys?

Hello, I am trying to build QE 6.1 (thank you for the previous suggestion about using ELPA 2016), however I am running into some problems with the build still.

It seems like building with configure-qe-hsw-omp.sh tries to use the gnu C compiler instead of icc (not sure if this is correct), and then it seems like there are some issues linking to F77 libraries, and the configure script looks for a make.sys file although QE 6.1 uses a make.inc file. I have attached the output of the configure script so you can see.

Running make pw -j immediately after the configure step yields the following output:

qe_6.1$ make pw -j
Makefile:9: make.inc: No such file or directory
make: *** No rule to make target `make.inc'.  Stop.

For your reference, I am building QE 6.1 on an Ubuntu 14.04 LTS machine with a Intel(R) Xeon(R) CPU E5-2687W v4 CPU and I am using Intel Parallel Studio 2016 Update 4.

config_output.txt

Any help would be appreciated!

No need to patch QE for ELPA

There is no need to add patch __ELPA_2017 or __ELPA_2018 for QE 6.2.1 and 6.3.
By default the configure script set __ELPA_2016 which is compatible with ELPA 2017.11 and 2018.05

Nothing shows up after 20 mins

Hi
I did ./configure-get.sh but nothing shows up.
Do I need to load modules before hand? I am working on a HPC cluster.

Issue with QE v6.3 with and without openmp

Hi,
I have parallel studio 2017 update 7 and I have successfully compiled ELPA 2017.11.001 then QE v6.3 via the configure-xxx-hsw.sh script.
It is okay when I try to run QE v6.3 on a single node in the cluster, i.e. srun -p ABC -N 1 -n 176 pw.x < my.in > my.out.
However, once I try over 2 nodes, i.e. srun -p ABC -N 2 -n 352 pw.x < my.in > my.out, it produces the strange "Error in routine cdiaghg problems computing cholesky" error.
If I compile ELPA and QE with configure-xxx-hsw-omp.sh script, it is also okay for single node. However, if 2 nodes, it produces "PMPI_Group_incl: Invalid rank, error stack:" message in the slurm-xxx.out
Could you please have a look at QE v6.3?

Moreover, conventional compilation without xconfigure run okay across multi nodes, i.e. ./configure CC=icc CXX=icpc F77=ifort F90=ifort MPIF90=mpiifort --enable-shared --enable-parallel --disable-openmp --with-scalapack=intel CFLAGS="-O3 -I -xCORE-AVX2" CXXFLAGS="-O3 -I -xCORE-AVX2" FCFLAGS="-O3 -I -xCORE-AVX2" F90FLAGS="-O3 -I -xCORE-AVX2" FFLAGS="-O3 -I -xCORE-AVX2

Thanks,
Rolly

CP2K Compilation error at ".inject.sh: 1: Bad substitution"

LIBXSMM release-1.16.1 (Linux)

Intel Compiler: icc 2021.1, icpc 2021.1, and mpiifort 2021.1
C / C++ target: -xHost -qopt-zmm-usage=low
Fortran target: -xHost -qopt-zmm-usage=low

Intel VTune Amplifier support has been detected (enable with SYM=1).

make[4]: Nothing to be done for 'libxsmm'.
/home/luca/cp2k/cp2k-7.1/tools/build_utils/fypp -n /home/luca/cp2k/cp2k-7.1/exts/dbcsr/src/core/dbcsr_array_types.F dbcsr_array_types.F90
/home/luca/cp2k/cp2k-7.1/.inject.sh dbcsr_ -DNDEBUG mpiifort -c -align array64byte -complex-limited-range -free -fpp -threads -qopenmp-simd -qopt-matmul -qopenmp -O2 -xHost -qopt-zmm-usage=low -D__NO_IPI_DRIVER -D__parallel -D__SCALAPACK -D__MPI_VERSION=3 -D__INTEL -D__LIBINT -D__MAX_CONTR=4 -D__LIBXC -D__PLUMED2 -D__ELPA -D__LIBXSMM=1 -D_OPENMP_SIMD -D__MKL -D__FFTW3 -I/home/luca/cp2k/cp2k-7.1/obj/Linux-x86-64-intelx/psmp/exts/dbcsr -I/home/luca/cp2k//libint/intel-skx/include -I/home/luca/cp2k//libxc/intel-skx/include -I/home/luca/cp2k//elpa/intel-skx-omp/include/elpa_openmp-2020.11.001/elpa/ -I/home/luca/cp2k//elpa/intel-skx-omp/include/elpa_openmp-2020.11.001/elpa/../modules -I/home/luca/cp2k/cp2k-7.1/lib/Linux-x86-64-intelx/psmp/libxsmm/include -I/opt/intel/oneapi/mkl/latest/include -I/opt/intel/oneapi/mkl/latest/include/fftw -diag-disable 8290,8291,10010,10212,10411,11060 -D__SHORT_FILE__=""dbcsr_array_types.F"" -I'/home/luca/cp2k/cp2k-7.1/exts/dbcsr/src/core/' -I'/home/luca/cp2k/cp2k-7.1/exts/dbcsr/src' dbcsr_array_types.F90
/home/luca/cp2k/cp2k-7.1/.inject.sh: 1: Bad substitution
/home/luca/cp2k/cp2k-7.1/.inject.sh: 1: Bad substitution
make[4]: *** [/home/luca/cp2k/cp2k-7.1/exts/dbcsr/Makefile:429: dbcsr_array_types.o] Error 2
make[3]: *** [Makefile:129: libdbcsr] Error 2
make[2]: *** [/home/luca/cp2k/cp2k-7.1/exts/Makefile.inc:16: dbcsr] Error 2
make[1]: *** [/home/luca/cp2k/cp2k-7.1/Makefile:123: psmp] Error 2
make: *** [Makefile:118: all] Error 2

CP2K build fails

See cp2k/cp2k#1106

How to run c tests?

Hi,
I know it may seem as a novice question but I am having difficulties running the C tests.
If I am not mistaken I need to compile the tests inside the /test/C folder right? how do I compile them?
I have installed elpa using https://xconfigure.readthedocs.io/en/latest/elpa/

I would really love the help. thanks in advance!

Configure scripts for QE build with OpenMP for non-OpenMP configuration

I had a problem running QE 6.0 on multiple nodes on our cluster after configuring with "configure_qe_snb.sh", and I think it's because the code is being built with OpenMP. Looking at that particular configure script, it seems that it is indeed being built with OpenMP. Can this be updated? Am I correct that OpenMP prevents codes being run on multiple nodes?

cp2k/README.md: "./" missing in front of configure-lib*-skx-gnu.sh

Please add the "./" in front of configure-libxc-skx-gnu.sh and configure-libint-skx-gnu.sh in cp2k/Readme.md.

Is there a way to configure ELPA installation for Ivy Bridge processors (AVX instruction set)?

I am trying to build QE 6.0 for a Linux cluster with Intel Xeon E5-2680 v2 processors using Intel MPI, and I see that I can configure QE for these CPUs with "configure-qe-snb.sh", but there is not a corresponding "configure-elpa-snb.sh". I was just curious if this is something that you would be adding soon, or if it is not possible to build elpa with the AVX instruction set? Thank you for putting this project together, it is really helpful (I am not an HPC expert by any means). If I have misunderstood something, I would really appreciate it if you could provide some clarification.

Feature Request

Hello Hans,
I like your CP2K xconfigure very much, and I have been deployed your version CP2K in my cluster for my group. Recently, I found the Pole EXpansion and Selected Inversion (PEXSI) may help improved the CP2K performance. Could you add this package to your xconfigure? I aware it may take time and hard work, so any change and advise I will be very appreciate.

Best Regards,
Linfeng

Lost compatibility with QE 6.1

Hello,

what if I want to run again xconfigure with QE 6.1? It was possible in the past I believe, not it fails.

Thanks -- FS

nonexistent url in configure-get.sh file

I am trying to use xconfigure to install elpa-2017.05.001.rc1 (prior to installing QE v.6.1), but the configure-get.sh file is calling to the url https://github.com/hfp/xconfigure/raw/master/elpa/.filelist, which does not exist in the git repository. Is this something that is supposed to be present or is it deprecated / not necessary anymore?

configure in libint-v2.6.0-cp2k-lmax-6

The "configure" in the libint-v2.6.0-cp2k-lmax-6 doesn't support the following options:
" --with-cc-optflags, --with-libderiv-max-am1, --with-libint-max-am, --disable-libtool".
That is to to say:

The script files obtained by "configure-get.sh libint":
configure-libint-hsw-gnu.sh
configure-libint-hsw.sh
configure-libint-knl.sh
configure-libint.sh
configure-libint-skx-gnu.sh
configure-libint-skx.sh
configure-libint-snb.sh
If we run one of them, the following waring message will appear in the end:
"configure: WARNING: unrecognized options: --with-cc-optflags, --with-libderiv-max-am1, --with-libint-max-am, --disable-libtool"

How could I get the correct "configure" script file?

QE - Compilation fails using configure-qe-skx

It seems that the make.inc generated from configure-qe-skx.sh still refers to threaded MKL. If I compile pw using that make.inc file, I got a lot of errors like:
/opt/intel/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_intel_thread.a(dpttrs_par.o): In functionmkl_lapack_dpttrs':
dpttrs_omp_gen.f:(.text+0x177): undefined reference to __kmpc_global_thread_num' dpttrs_omp_gen.f:(.text+0x189): undefined reference to __kmpc_ok_to_fork'
dpttrs_omp_gen.f:(.text+0x1a2): undefined reference to __kmpc_push_num_threads' dpttrs_omp_gen.f:(.text+0x1ea): undefined reference to __kmpc_fork_call'
dpttrs_omp_gen.f:(.text+0x205): undefined reference to __kmpc_serialized_parallel' dpttrs_omp_gen.f:(.text+0x26d): undefined reference to __kmpc_end_serialized_parallel'
dpttrs_omp_gen.f:(.text+0x34a): undefined reference to omp_get_thread_num' dpttrs_omp_gen.f:(.text+0x354): undefined reference to omp_get_num_threads'
`

Adding the option -qopenmp to the variable LDFLAGS solves the error, but I guess that in that case QE still uses the threaded version of MKL.

Moreover, the configure-qe-skx.sh script adds the option -threads to the FFLAGS variable, which does not seem necessary.

New feature request

Is it possible to enable these interfaces when compiling cp2k with intel compiler?

1. interface to PEXSI
2. interface to QUIP
3. interface to SIRIUS, which depends on gsl, libsgp, elpa, hdf5, etc.

Build fails in make all

started with compiler environment 17.0.4.196
Build elpa-2017.11.001 default-hsw-omp with no problem.
then configure-qe-hsw-omp.sh
then qe-6.2 'make all' fails in LAXlib with:

dspev_drv.f90(708): error #6404: This name does not have a type, and must have an explicit type.   [GET_ELPA_ROW_COL_COMMS]
     ierr = get_elpa_row_col_comms(ortho_comm, my_prow, my_pcol,mpi_comm_rows, mpi_comm_cols)
------------^

neb.x in Quantum ESPRESSO fails to compile with no error message

Using the configure scripts for Sandy Bridge processors, we compiled both ELPA and Quantum ESPRESSO v.6.0, and see that pw.x builds perfectly well. However, when building the neb.x code, the executable fails to be built despite the console reporting zero warnings (to my knowledge) and no apparent error messages. We also see an empty fortran source file appear in the NEB/src directory called "init_us_1.f90". Any ideas about what could be causing this?

Issue with the arch file of CP2K

When compiling the development version of CP2K with the intel compiler and Linux-x86-64-intelx arch, an error occurs:

/srv/cp2k/src/mp2_eri.F(32): error #7002: Error in opening the compiled module file. Check INCLUDE paths. [DBCSR_API]

Including the path to dbcsr library in line 676 of the arch file may be a solution.

error in cp2k compilation

I'm trying to compile cp2k. I use Intel 2018u4 and gcc 8.2, the cp2k is in the master branch. When I run

./configure-libint-skx-gnu.sh

for libint, it reports:
...
Skipping ./src/lib/libint/Makefile
Skipping ./src/lib/libr12/Makefile
configure: creating ./config.status
./configure: line 16238: : command not found
./configure: line 16238: no: command not found
./configure: line 16238: : command not found
./configure: line 16238: : command not found
./configure: line 16238: x86_64-unknown-linux-gnu: command not found
./configure: line 16238: : command not found
./configure: line 16238: linux-gnu: command not found
./configure: line 16238: : command not found
./configure: line 16238: : command not found
./configure: line 16238: x86_64-unknown-linux-gnu: command not found
./configure: line 16238: : command not found
./configure: line 16238: linux-gnu: command not found
./configure: line 16238: : command not found
Usage: /usr/bin/grep [OPTION]... PATTERN [FILE]...
Try '/usr/bin/grep --help' for more information.
./configure: line 16238: : command not found
./configure: line 16238: /usr/bin/grep -E: No such file or directory
../configure: line 16238: : command not found
./configure: line 16238: ln -s: command not found
./configure: line 16238: : command not found
./configure: line 16238: o: command not found
./configure: line 16238: : command not found
/usr/bin/ar: no operation specified
Usage: ranlib [options] archive
Generate an index to speed access to archives
The options are:
@ Read options from
--plugin Load the specified plugin
-D Use zero for symbol map timestamp
-U Use actual symbol map timestamp (default)
-t Update the archive's symbol map timestamp
-h --help Print this help message
-v --version Print version information
ranlib: supported targets: elf64-x86-64 elf32-i386 elf32-iamcu elf32-x86-64 a.out-i386-linux pei-i386 pei-x86-64 elf64-l1om elf64-k1om elf64-little elf64-big elf32-little elf32-big plugin srec symbolsrec verilog tekhex binary ihex
./configure: line 16238: : command not found
gcc: fatal error: no input files
compilation terminated.
./configure: line 16238: : command not found
./configure: line 16238: -O3 -mavx512f -mavx512cd -mavx512dq -mavx512bw -mavx512vl -mfma : command not found
...

Then I use configure-libint-hsw-gnu.sh, the compilation is successful. But when I run the cp2k exe file, it reports:
Program received signal SIGILL: Illegal instruction.

Backtrace for this error:
#0 0x2b6ce27cc27f in ???
#1 0x2a49e52 in ???
#2 0x408a8c in ???
#3 0x2b6ce27b83d4 in ???
#4 0x408abc in ???
#5 0xffffffffffffffff in ???
Illegal instruction

YAMBO?

It would be AWESOME if someone wrote a configure recipe for YAMBO

Compilation instructions for AMD EPYC

Could you provide the compilation guidance for AMD EPYC?

I have tried to use standard flags with Intel Compiler ver. 2018.
https://xconfigure.readthedocs.io/en/latest/cp2k/#build-the-intel-fork-of-cp2k

After the compilation with Intel Compiler series 18 using Intel tool chain, I get an error:

[azure@ip-0A000104 cp2k]$ ./exe/Linux-x86-64-intelx/cp2k.sopt Please verify that both the operating system and the processor support Intel(R) X87, CMOV, MMX, FXSAVE, SSE, SSE2, SSE3, SSSE3, SSE4_1, SSE4_2, MOVBE, POPCNT, F16C, AVX, FMA, BMI, LZCNT, AVX2, AVX512DQ, AVX512F, ADX, AVX512CD, AVX512BW, AVX512VL and CLWB instructions.