Giter Site home page Giter Site logo

alidist's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

alidist's Issues

Investigate error in alibuild while assigning revisions to a package.

Looks like under certain circumstances one can get errors like:

Package AliEn-Runtime with hash 55f057ced71040939e96078d881f5a7781302590 is already found in /build/workarea/sw/20150829/TARS/slc5_x86-64/AliEn-Runtime/AliEn-Runtime-v2-19-xrd2-2.slc5_x86-64.tar.gz. Not building.
Package CMake with hash 1cbdcfcebb341cdf548808ececd312d36477447f is already found in /build/workarea/sw/20150829/TARS/slc5_x86-64/CMake/CMake-v2.8.12-1.slc5_x86-64.tar.gz. Not building.
Traceback (most recent call last):
  File "alibuild/aliBuild", line 429, in ?
    spec["revision"] = min(set(range(1, max(busyRevisions)+2)) - set(busyRevisions))
ValueError: max() arg is an empty sequence

see for example:

https://alijenkins.cern.ch/job/build-any-ib/948/console

Tests for ThePEG

We currently have the thepeg-test.sh recipe containing a simple generation done with ThePEG, based on the examples @qgp provided me with some time ago.

We would like to extend this test by adding some plots we can check to understand that everything is correct, and ultimately merge it with the aliroot-tests.sh recipe, so that they will become visible on ali-ci.cern.ch.

If you look at aliroot-tests.sh you'll notice that only .root, .sql and .log files are copied to the destination.

@qgp, could you please take care of it? We think this will be useful for everybody :-)

Check that cmake was not built with gcc 5.2.0

If you run on a mixed GCC 5.2.0 / GCC 4.9.3 environment, e.g. on some Ubuntu. Using the system cmake will not work, because it will fail as soon as we enable our own GCC. Solution is to simply build our own cmake when the system does not work.

AliBuild failing for yaml-cpp-latest on SLC6 with default boost installed

Setup : SLC6 with boost 1.41 installed via yum

Package to build : O2

Command used : ./alibuild/aliBuild build o2

Boost 1.59 is built by alibuild (presumabily because 1.41 is too old). This works ok. But when building yaml-cpp it fails to identify properly the boost installation to use (content of /home/flpprotodev/alice/sw/BUILD/yaml-cpp-latest/log) :

  • case $ARCHITECTURE in

  • cmake /home/flpprotodev/alice/sw/SOURCES/yaml-cpp/v0.5.2/release-0.5.2 -DCMAKE_INSTALL_PREFIX:PATH=/home/flpprotodev/alice/sw/INSTALLROOT/d90e919deff677cdfdf7f8007d58d6a4c777b368/slc6_x86-64/yaml-cpp/v0.5.2-1 -DBUILD_SHARED_LIBS=YES -DBOOST_ROOT:PATH=/home/flpprotodev/alice/sw/slc6_x86-64/boost/v1.59.0-1 -DCMAKE_SKIP_RPATH=YES -DSKIP_INSTALL_FILES=1
    -- The C compiler identification is GNU 4.9.1
    -- The CXX compiler identification is GNU 4.9.1
    -- Check for working C compiler: /opt/rh/devtoolset-3/root/usr/bin/cc
    -- Check for working C compiler: /opt/rh/devtoolset-3/root/usr/bin/cc -- works
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Detecting C compile features
    -- Detecting C compile features - done
    -- Check for working CXX compiler: /opt/rh/devtoolset-3/root/usr/bin/c++
    -- Check for working CXX compiler: /opt/rh/devtoolset-3/root/usr/bin/c++ -- works
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    CMake Error at /usr/lib64/boost/Boost.cmake:536 (message):
    The imported target "boost_date_time-static" references the file

    "/usr/lib64/lib64/libboost_date_time.a"

    but this file does not exist. Possible reasons include:

    • The file was deleted, renamed, or moved to another location.

    • An install or uninstall procedure did not complete successfully.

    • The installation package was faulty and contained

      "/usr/lib64/boost/Boost.cmake"

    but not all the files it references.

Call Stack (most recent call first):
/usr/lib64/boost/BoostConfig.cmake:28 (include)
/opt/hackathon_dependencies/share/cmake-3.3/Modules/FindBoost.cmake:210 (find_package)
CMakeLists.txt:116 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/flpprotodev/alice/sw/BUILD/d90e919deff677cdfdf7f8007d58d6a4c777b368/yaml-cpp/CMakeFiles/CMakeOutput.log".

To be noted that BOOST_ROOT is correct but Boost_DIR and Boost_INCLUDE_DIR are not resulting to a mixed config.

Various issues with Alien-runtime

  • repository should not be under dberzano.
  • fails to build on mac with the following. It can be worked around by dropping the buildtime target from the Makefile.
DEBUG:AliPhysics:7bda97d1: svn: E155007: '/Users/ktf/work/active/sw/BUILD/55f057ced71040939e96078d881f5a7781302590/AliEn-Runtime/apps/tools/eugridpma-carep-classic' is not a working copy
DEBUG:AliPhysics:7bda97d1: date: illegal time format
DEBUG:AliPhysics:7bda97d1: usage: date [-jnu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
DEBUG:AliPhysics:7bda97d1:             [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
DEBUG:AliPhysics:7bda97d1: ERROR getting Makefile time
DEBUG:AliPhysics:7bda97d1: make[2]: *** [buildtime] Error 1
DEBUG:AliPhysics:7bda97d1: make[1]: *** [dep-../../..//apps/tools/eugridpma-carep-classic] Error 2
DEBUG:AliPhysics:7bda97d1: make: *** [dep-../..//apps/alien/alien-ca] Error 2
DEBUG:AliPhysics:7bda97d1:
  • In various places cp -d is used in place of cp -P (or even just cp)
  • Does not compile with clang due to the fact -lgcc is specified.

Spurios warnings from the zlib packaged in Alien-runtime

@dberzano looks like the zlib which we have in Alien-runtime is not compiled correctly. I get a bunch of warnings:

DEBUG:AliRoot:1d5515ed: /build/workarea/sw/20150829/slc7_x86-64/CMake/v2.8.12-1/bin/cmake: /build/workarea/sw/20150829/INSTALLROOT/55f057ced71040939e96078d881f5a7781302590/slc7_x86-64/AliEn-Runtime/v2-19-xrd2-2/lib/libz.so.1: no version information available (required by /build/workarea/sw/20150829/slc7_x86-64/CMake/v2.8.12-1/bin/cmake)

while compiling various things.

See for example:

https://alijenkins.cern.ch/job/build-any-ib/950/console

Separate python itself from its modules

Right now its very likely we rebuild python simply because one of the required module dependencies are not there, e.g. certifi. We should really split the python package in two:

  • Python
  • Python-environment

where Python-environment is simply building the missing modules.

Build DDS using alibuild

  • Make sure cmake can find BOOST:
DEBUG:zlib:8822efa6: CMake Error at /Users/ktf/work/active/sw/osx_x86-64/CMake/v2.8.12-3/share/cmake-2.8/Modules/FindBoost.cmake:1111 (message):
DEBUG:zlib:8822efa6:   Unable to find the requested Boost libraries.
DEBUG:zlib:8822efa6:
DEBUG:zlib:8822efa6:   Boost version: 1.57.0
DEBUG:zlib:8822efa6:
DEBUG:zlib:8822efa6:   Boost include path:
DEBUG:zlib:8822efa6:   /Users/ktf/work/active/sw/INSTALLROOT/a2418cba13efe7b8e8b58b5602ff359247208a78/osx_x86-64/boost/v1.57.0-5/include

JEWEL does not compile with gfortran 4.1.2

Hello Jochen,

gfortran -g -static  -c -o jewel-2.0.2.o jewel-2.0.2.f
 In file jewel-2.0.2.f:119

       write(logfid,*)'number of extrapolations in splitting integral: '
                                                                       1
Error: Syntax error in WRITE statement at (1)
 In file jewel-2.0.2.f:1235

     &'colour singlet system, will discard event'
      1
Error: Syntax error in WRITE statement at (1)
 In file jewel-2.0.2.f:1235

     &'colour singlet system, will discard event'
     1
Error: Bad continuation line at (1)
 In file jewel-2.0.2.f:1235

     &'colour singlet system, will discard event'
     1
Error: Unclassifiable statement at (1)
 In file jewel-2.0.2.f:1258

     &'colour singlet system, will discard event'
      1
Error: Syntax error in WRITE statement at (1)
 In file jewel-2.0.2.f:1258

     &'colour singlet system, will discard event'
     1
Error: Bad continuation line at (1)
 In file jewel-2.0.2.f:1258

     &'colour singlet system, will discard event'
     1
Error: Unclassifiable statement at (1)
 In file jewel-2.0.2.f:1315

     &'colour singlet system, will discard event'
      1
Error: Syntax error in WRITE statement at (1)
 In file jewel-2.0.2.f:1315

     &'colour singlet system, will discard event'
     1
Error: Bad continuation line at (1)
 In file jewel-2.0.2.f:1315

     &'colour singlet system, will discard event'
     1
Error: Unclassifiable statement at (1)
make: *** [jewel-2.0.2.o] Error 1

As we don't have our own compiler on the Grid yet, this must work with gfortran from GCC 4.1.2 on SLC5.

alibuild failing on FairRoot (Mac OS X)

Another problem, this time in the cmake stage for FairRoot (will have the same one for AliceO2 I guess) :

Looks like one should use -DUSE_DIFFERENT_COMPILER=TRUE on Mac...

-- fairsoft-config found
-- C compiler used for FairSoft installation:  /usr/bin/clang
-- C compiler used now:  /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-- CXX compiler used for FairSoft installation:  /usr/bin/clang++
-- CXX compiler used now:  /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-- The compiler during the compilation of FairSoft is different from the current one.
CMake Error at cmake/modules/CheckCompiler.cmake:64 (Message):
  This is seen as an error.  If you know that the setting is correct you can
  silence the error by using the CMake flag -DUSE_DIFFERENT_COMPILER=TRUE
Call Stack (most recent call first):
  CMakeLists.txt:55 (Check_Compiler)


-- Configuring incomplete, errors occurred!
See also "/Users/laurent/o2/o2work/sw/BUILD/c8cb7759bd42965e13d59d304223f25e19b0aab1/FairRoot/CMakeFiles/CMakeOutput.log".

Include our own BZIP2

It looks like we libbz2-dev is not installed by default on ubuntu. We should decide whether or not we should include it or add specific instructions about installing it by hand.

alibuild failing on Python (Mac OS X)

Hi,

Trying to install on Mac OS X (Darwin 15.3.0 Darwin Kernel Version 15.3.0: Thu Dec 10 18:40:58 PST 2015; root:xnu-3248.30.4~1/RELEASE_X86_64 x86_64), I fail on Python. Attached the result of

alibuild/aliBuild -d build O2

failbuild.txt

and :

python -c 'import matplotlib; import numpy; import certifi ; import sys; sys.exit(1 if sys.version_info < (2, 7) else 0)'
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named certifi

using, as instructed :

sudo pip install matplotlib
sudo pip install numpy
sudo pip install certifi

got me past those issues.

Package python 2.7

Python 2.4 or newer is required for YODA -> Rivet -> ThePEG (and other generators depending on Rivet). It is not available on SLC 5 and the lack of it is blocking the rebuild of Rivet. According to @ktf, it should be packaged.

Create own repository for fastjet

Fastjet and fastjet contrib are currently picked up via curl. A better way would be to have our own alisw/FastJet repository and have them committed there.

Compilation problem with FairRoot (when trying to build O2) on Mac OS 10.11.4

Hi Giulio,
I'm trying to build O2 using:
IB/v5-08/next
and asking for a specific version of aliroot:
AliRoot@feature-muonhlt,O2

I build with command:
../../alibuild/aliBuild -z -w ../../sw --disable GEANT4,GEANT4_VMC,HepMC,simulation,fastjet,generators,DDS build O2

But, during the compilation of the dependencies, FairRoot crashes complaining about missing libHist.5.so (see attached log).

log.txt

Also, I noticed in the log that the GEANT3_DIR is not set by CMake...although I'm not disabling GEANT3 (at the beginning I disabled it...but thought that fairroot needed it since in the build.sh the GEANT3_DIR seems to be explicitly required, so I started again from scratch and did not disabled GEANT3 this time...).

My environment is:
Mac OS 10.11.4
with:
Xcode 7.3
clang: Apple LLVM version 7.3.0 (clang-703.0.29)
cmake 3.5.2

By the way, I disable simulation+generators first of all because in the end I only need the reconstruction part...but also because at some point I tried to enable generators, and this triggers the compilation of python which fails on Mac...

Thanks in advance,
ciao,
Diego

Alien-runtime seems to be compiled without -fPIC

It looks like Alien-runtime is compiled without -fPIC and this causes errors on Ubuntu while compiling root:

DEBUG:AliRoot:1d5515ed: /usr/bin/ld: /build/workarea/sw/20150829/INSTALLROOT/55f057ced71040939e96078d881f5a7781302590/ubt14_x86-64/AliEn-Runtime/v2-19-xrd2-2/lib/libz.so.1: no version information available (required by /usr/lib/libbfd-2.24-system.so)
DEBUG:AliRoot:1d5515ed: /usr/bin/ld: lib/liblzma.a(liblzma_la-common.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
DEBUG:AliRoot:1d5515ed: lib/liblzma.a: error adding symbols: Bad value
DEBUG:AliRoot:1d5515ed: collect2: error: ld returned 1 exit status
DEBUG:AliRoot:1d5515ed: make: *** [lib/libCore.so] Error 1
DEBUG:AliRoot:1d5515ed: make: *** Waiting for unfinished jobs....
DEBUG:AliRoot:1d5515ed: as: /build/workarea/sw/20150829/INSTALLROOT/55f057ced71040939e96078d881f5a7781302590/ubt14_x86-64/AliEn-Runtime/v2-19-xrd2-2/lib/libz.so.1: no version information available (required by /usr/lib/libbfd-2.24-system.so)

Full results at:

https://alijenkins.cern.ch/job/build-any-ib/957/console

ThePEG version bump

We want to move to ThePEG 20150811 in conjunction with PYTHIA 8.211_prelease. Can we make ThePEG depend on that version and then use it together with AliPhysics? How can we make sure there is no clash of PYTHIA versions?

PS: I know that this is ugly but we miss features without the special PYTHIA version for ThePEG.

More externals to pick up from the system

We should pick from the system:

  • GSL

Moreover we should make sure that the user knows about having to install:

  • bison
  • flex
  • libperl-dev

by pre-emptly checking when building their dependencies.

wrong path for G4INSTALL_DATA

the location in G4INSTALL_DATA is invalid.

It should be "$GEANT4_ROOT/share/Geant4-10.1.3" instead if "$GEANT4_ROOT/share/Geant4-10.1.2"

avoid verbose debug messages of alien client when building with defaults-debug.sh

Using a build of aliroot with defaults-debug.sh I get in my test reconstruction tons of messages like the ones below from the alien client, which is quite annoying (keys and tokens changed to protect the innocents ;-) )

(DEBUG_GCLIENT=5,InternalCommand): fStoreToken is 0
(DEBUG_GCLIENT=8,InternalCommand): codeccommand: >accessread/alice/data/2011/OCDB/GRP/CTP/Aliases/Run0_999999999_v1_s0.rootunknown01<
(DEBUG_GCLIENT=8,InternalCommand): pwd/alice/cern.ch/user/l/laphecet/remoteDebug0n(DEBUG_GCLIENT=7,InternalCommand): b64 enc key: >2IvzAm6yyvq25+CmmB1mCvyA==<   decoded key length 16
(DEBUG_GCLIENT=5,InternalCommand): ENCRYPTING MAGIC TOKEN
(DEBUG_GCLIENT=10,InternalCommand): ecrypted magic token: >PgrLZ+bSVUX2s6AugpghKZJ+yJtFpwbiuavExvO80h9sGyojKz+E+HQ5OTwDYa1lQ==< (64)
(DEBUG_GCLIENT=8,InternalCommand): Encrypted command:>8nLmp7/nvBeWKZX4jT86eai6BvjqIGasuxWRRMZ54FeZXpTR8EYgDBFA1Ns0F64mWCT07K3xWu8TThL6txsPosaeWR5rtYP25pCih1D1VbQ8+bPa1APcSPTgDln1860hqAv2Y7UGK2yWOfRvl4OS3hD/6eR9Nf7udrHBARi6nv4rdZv7IgC8DY7bvWdzpI+oIzgmUxh3+gh14ygmgdg==<    length:212
160407 09:52:49 8971 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8 bytes).
(DEBUG_GCLIENT=1,InternalCommand): Timing: t2 13 microsec
(DEBUG_GCLIENT=1,InternalCommand): Timing: t3 9 microsec
(DEBUG_GCLIENT=1,InternalCommand): Timing: t4 5 microsec
(DEBUG_GCLIENT=1,InternalCommand): Timing: t5 10 microsec
(DEBUG_GCLIENT=1,InternalCommand): Timing: t6 17 microsec
(DEBUG_GCLIENT=1,InternalCommand): Timing: t7 53 microsec
(DEBUG_GCLIENT=1,InternalCommand): Timing: t8 277040 microsec
(DEBUG_GCLIENT=1,InternalCommand): Timing: t0 277147 microsec
(DEBUG_GCLIENT=5,InternalCommand): DECRYPTING SERVER'S MAGIC TOKEN: OyHDHGGIs8mhHyokGT4HRkeVk89Zx1QhQ1thaEC7s3jzv+1hgUfbr5Hnj71v60lVzart
(DEBUG_GCLIENT=5,InternalCommand): magic token is: cowardly hiding two red horses
(DEBUG_GCLIENT=5,InternalCommand): codec decoding returned status=1
(DEBUG_GCLIENT=8,InternalCommand): client PWD set to >/alice/cern.ch/user/l/laphecet/<
(DEBUG_GCLIENT=8,InternalCommand): returning from command >access read /alice/data/2011/OCDB/GRP/CTP/Aliases/Run0_999999999_v1_s0.root unknown 0 1<
I-TAlienFile::Open: Accessing image 1 of alien:///alice/data/2011/OCDB/GRP/CTP/Aliases/Run0_999999999_v1_s0.root in SE <ALICE::CERN::OCDB>

Add check for csrutil on mac

Given that we do not support compiling with SIP enabled we should have a system_requirement check on mac which makes sure that SIP is disabled.

ALICE O2 build fails on CC7 with python 2.7.10 error (freetype)

See log.

Also, the message "Building our own python. If you want to avoid this, please install python >= 2.7" is strange since we have in the system python 2.7.8 enabled (via devtoolset).

[vasco@aidrefflp01 o2Tutorial]$ python --version
Python 2.7.8
[vasco@aidrefflp01 o2Tutorial]$

ThePEG crashes in LEGO train and on lxplus

@dberzano, @mfloris: The new version of ThePEG segfaults in the LEGO trains and on lxplus. I just recompiled everything in the docker container and there it works. Any idea what is different for the LEGO train or lxplus installation? The relocation is already done for the installation in the container, right?

Texinfo needed to compile root on SLC6

Building [email protected] it fails :

WARNING: makeinfo' is missing on your system. You should only need it if you modified a.texi' or .texinfo' file, or any other file indirectly affecting the aspect of the manual. The spurious call might also be the consequence of using a buggymake' (AIX,
DU, IRIX). You might want to install the Texinfo' package or theGNU make' package. Grab either from any GNU archive site.
make[3]: *** [bfd.info] Error 1

It might be related to #251 where the same dependency was mentioned.

Do Jenkins build in same work-area

This way we should minimise the amount of redownloading / rebuilding. Notice however we should not reuse the same as the standard builds, otherwise we end up uploading the wrong thing.

PROOF does not compile on Ubuntu

Chiara's configuration fails with:

DEBUG:AliPhysics:0: stressProof.cxx:135:28: fatal error: proof/getProof.C: No such file or directory
DEBUG:AliPhysics:0:  #include "proof/getProof.C"
DEBUG:AliPhysics:0:                             ^
DEBUG:AliPhysics:0: compilation terminated.
DEBUG:AliPhysics:0: Makefile:830: recipe for target 'stressProof.o' failed
DEBUG:AliPhysics:0: make: *** [stressProof.o] Error 1
DEBUG:AliPhysics:0: make: *** Waiting for unfinished jobs....
DEBUG:AliPhysics:0:
ERROR:AliPhysics:0: Error while executing /data/alisw/sw/SPECS/ubuntu1504_x86-64/ROOT/v5-34-30-alice-1/build.sh
ERROR:AliPhysics:0: Log can be found in /data/alisw/sw/BUILD/ROOT-latest/log

waiting for her to provide the list of packages installed on her system.

Document the branch structure and switch integration builds to proper branches

Since a few days a number of branches have appeared in alidist. We need to document what they are for and eventually select which one to run in the integration builds (which are currently using master).

  • Add documentation about branches nomeclature somewhere (alidist/README.md?)
  • Fix ali-bot/config.yaml to pick up the proper branches.

Missing dependencies when building O2 on slc6

While building O2 on slc6, I had to manually install the following RPMs:

  • texinfo (missing makeinfo error while building MPFR)
  • perl-ExtUtils-Embed (missing while building xalienfs)

Implement a `--prefer-system` option

The idea is to have an additional entry in the YAML part of the recipe which will allow for testing the compatibility of an eventual system version with the recipe itself. Then, if a to-be-introduced --prefer-system option is passed, alibuild will perform the check, assign an hash to the system version and disable our own version of the tool. This is in particular useful for things which are common on most systems, like zlib, cmake, or even GCC.

GEANT_VMC tests

It would be great to have small GEANT_VMC tests being available so that we can use them to quickly tests PRs when GEANT changes. What we need for that is:

  • A geant-test.sh recipe which depends on all the required parts. (see #360)
  • Add the rest of the tests to geant-test.sh
  • Update jenkinsfile to actually run the tests
  • Right now geant-test.sh should use pythia6.sh since the one in AliRoot does not have all the symbols, however in the mid-term we should make sure the two converge to be the same.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.