alibuild failing on FairRoot (Mac OS X)

[alisw]

alibuild failing on FairRoot (Mac OS X)

[ktf]

Do you have any ALFA related environment variable set?

[ktf]

Or SIMPATH, as a matter of fact. I think this is bad interaction with the old installation procedure.

[aphecetche]

Hum, actually, yes, from a previous (non alibuild) installation...

15:54:50 laurent@nanpc420:/o2/o2work> env | grep FAIR
FAIRROOTPATH=/Users/laurent/o2/alfa/inst
15:55:25 laurent@nanpc420:/o2/o2work> env | grep SIM
SIMPATH=/Users/laurent/o2/alfa/inst

Removing those I'm now failing in DDS

DEBUG:AliRoot:5c8d09ae: Scanning dependencies of target dds-agent
DEBUG:AliRoot:5c8d09ae: Scanning dependencies of target dds-commander
DEBUG:AliRoot:5c8d09ae: [ 80%] Building CXX object dds-agent/CMakeFiles/dds-agent.dir/src/main.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 81%] Building CXX object dds-commander/CMakeFiles/dds-commander.dir/src/main.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 82%] Linking CXX executable ui-custom-cmd
DEBUG:AliRoot:5c8d09ae: [ 83%] Linking CXX executable task-custom-cmd
DEBUG:AliRoot:5c8d09ae: [ 83%] Built target ui-custom-cmd
DEBUG:AliRoot:5c8d09ae: [ 84%] Building CXX object plugins/dds-submit-ssh/CMakeFiles/dds-submit-ssh.dir/src/worker.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 84%] Built target task-custom-cmd
DEBUG:AliRoot:5c8d09ae: [ 85%] Building CXX object dds-commander/CMakeFiles/dds-commander.dir/src/ConnectionManager.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 86%] Linking CXX executable task-type-one
DEBUG:AliRoot:5c8d09ae: [ 86%] Built target task-type-one
DEBUG:AliRoot:5c8d09ae: [ 87%] Building CXX object dds-agent/CMakeFiles/dds-agent.dir/src/CommanderChannel.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 88%] Linking CXX executable task-type-two
DEBUG:AliRoot:5c8d09ae: [ 88%] Built target task-type-two
DEBUG:AliRoot:5c8d09ae: [ 89%] Building CXX object dds-commander/CMakeFiles/dds-commander.dir/src/AgentChannel.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 90%] Building CXX object dds-agent/CMakeFiles/dds-agent.dir/src/UIChannel.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 91%] Linking CXX executable dds-submit-ssh
DEBUG:AliRoot:5c8d09ae: [ 91%] Built target dds-submit-ssh
DEBUG:AliRoot:5c8d09ae: [ 92%] Building CXX object dds-commander/CMakeFiles/dds-commander.dir/src/UIChannelInfo.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 93%] Linking CXX executable dds-custom-cmd
DEBUG:AliRoot:5c8d09ae: [ 93%] Built target dds-custom-cmd
DEBUG:AliRoot:5c8d09ae: [ 94%] Building CXX object dds-agent/CMakeFiles/dds-agent.dir/src/AgentConnectionManager.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 95%] Building CXX object dds-commander/CMakeFiles/dds-commander.dir/src/SSHScheduler.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 96%] Building CXX object dds-commander/CMakeFiles/dds-commander.dir/src/ChannelId.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 97%] Building CXX object dds-agent/CMakeFiles/dds-agent.dir/src/UIConnectionManager.cpp.o
DEBUG:AliRoot:5c8d09ae: [ 98%] Linking CXX executable dds-commander
DEBUG:AliRoot:5c8d09ae: [ 98%] Built target dds-commander
DEBUG:AliRoot:5c8d09ae: [100%] Linking CXX executable dds-agent
DEBUG:AliRoot:5c8d09ae: [100%] Built target dds-agent
DEBUG:AliRoot:5c8d09ae: + make wn_bin
DEBUG:AliRoot:5c8d09ae: [ 5%] Built target dds_user_defaults_lib
DEBUG:AliRoot:5c8d09ae: [ 60%] Built target dds_protocol_lib
DEBUG:AliRoot:5c8d09ae: [ 73%] Built target dds_intercom_lib
DEBUG:AliRoot:5c8d09ae: [ 89%] Built target dds-agent
DEBUG:AliRoot:5c8d09ae: [ 97%] Built target dds-user-defaults
DEBUG:AliRoot:5c8d09ae: Scanning dependencies of target wn_bin
DEBUG:AliRoot:5c8d09ae: [100%] Generate WN binary package
DEBUG:AliRoot:5c8d09ae: cp: /Users/laurent/o2/o2work/sw/BUILD/6fda8cf283fc0a562d9f4093cbcb1e0b674e7b76/DDS/dds-wrk-bin/libboost_log-mt.dylib: Permission denied
DEBUG:AliRoot:5c8d09ae: ERROR: Can not build DDS worker package. Missing dependency.
DEBUG:AliRoot:5c8d09ae: make[3]: *** [CMakeFiles/wn_bin] Error 1
DEBUG:AliRoot:5c8d09ae: make[2]: *** [CMakeFiles/wn_bin.dir/all] Error 2
DEBUG:AliRoot:5c8d09ae: make[1]: *** [CMakeFiles/wn_bin.dir/rule] Error 2
DEBUG:AliRoot:5c8d09ae: make: *** [wn_bin] Error 2
DEBUG:AliRoot:5c8d09ae:
ERROR:AliRoot:5c8d09ae: Error while executing /Users/laurent/o2/o2work/sw/SPECS/osx_x86-64/DDS/master-2/build.sh

[ktf]

I will force unset those in the build recipe to avoid mistakenly picking them up.

Regarding DDS, this is an issue we just spotted with Anar (see FairRootGroup/DDS#117). While I try to understand what is going on, can you simply disable DDS with --disable DDS?

[aphecetche]

Still an issue :

alibuild/aliBuild -j 1 -d --disable GEANT4_VMC,GEANT4,HepMC,GEANT3,DDS,fastjet,pythia build O2

...
DEBUG:AliRoot:e92c028e: [ 66%] Building CXX object fairmq/CMakeFiles/FairMQ.dir/options/FairProgOptions.cxx.o
DEBUG:AliRoot:e92c028e: [ 66%] Building CXX object fairmq/CMakeFiles/FairMQ.dir/options/FairMQProgOptions.cxx.o
DEBUG:AliRoot:e92c028e: [ 66%] Building CXX object fairmq/CMakeFiles/FairMQ.dir/options/FairMQParser.cxx.o
DEBUG:AliRoot:e92c028e: [ 67%] Linking CXX shared library ../lib/libFairMQ.so
DEBUG:AliRoot:e92c028e: ld: file not found: /Users/laurent/o2/o3work/sw/INSTALLROOT/4bcc1df7d350554d5d11ecd0c05aeedca5e736eb/osx_x86-64/sodium/v1.0.8-1/lib/libsodium.18.dylib for architecture x86_64
DEBUG:AliRoot:e92c028e: clang: error: linker command failed with exit code 1 (use -v to see invocation)
DEBUG:AliRoot:e92c028e: make[2]: *** [lib/libFairMQ.15.11.0.so] Error 1
DEBUG:AliRoot:e92c028e: make[1]: *** [fairmq/CMakeFiles/FairMQ.dir/all] Error 2
DEBUG:AliRoot:e92c028e: make: *** [all] Error 2
DEBUG:AliRoot:e92c028e:
ERROR:AliRoot:e92c028e: Error while executing /Users/laurent/o2/o3work/sw/SPECS/osx_x86-64/FairRoot/master-1/build.sh
Log can be found in /Users/laurent/o2/o3work/sw/BUILD/FairRoot-latest/log

[ktf]

Sorry for the late reply. I think I know what is going on but I've not the time to check this morning. In the meanwhile can you try doing the following:

git clone -b dev https://github.com/FairRootGroup/FairRoot

At the same level as alibuild and alidist and try again?

[aphecetche]

Hum, same thing I would say :

[ 56%] Building C object base/CMakeFiles/Base.dir/source/rclose.c.o
ld: file not found: /Users/laurent/o2/o3work/sw/INSTALLROOT/4bcc1df7d350554d5d11ecd0c05aeedca5e736eb/osx_x86-64/sodium/v1.0.8-1/lib/libsodium.18.dylib for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/libFairMQ.15.11.0.so] Error 1
make[1]: *** [fairmq/CMakeFiles/FairMQ.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 56%] Building C object base/CMakeFiles/Base.dir/source/swaplw.c.o
/Users/laurent/o2/o3work/sw/BUILD/960d7f0f83e6fe29408f15ac622a951e491697ff/FairRoot/base/G__BaseDict.cxx:6863:21: warning: reference cannot be bound to dereferenced null pointer in well-defined C++ code; pointer may be assumed to always convert to true [-Wundefined-bool-conversion]
         int R__n=(&R__stl) ? int(R__stl.size()) : 0;
                    ^~~~~~  ~
/Users/laurent/o2/o3work/sw/BUILD/960d7f0f83e6fe29408f15ac622a951e491697ff/FairRoot/base/G__BaseDict.cxx:6880:21: warning: reference cannot be bound to dereferenced null pointer in well-defined C++ code; pointer may be assumed to always convert to true [-Wundefined-bool-conversion]
         int R__n=(&R__stl) ? int(R__stl.size()) : 0;
                    ^~~~~~  ~
/Users/laurent/o2/o3work/sw/BUILD/960d7f0f83e6fe29408f15ac622a951e491697ff/FairRoot/base/G__BaseDict.cxx:6990:21: warning: reference cannot be bound to dereferenced null pointer in well-defined C++ code; pointer may be assumed to always convert to true [-Wundefined-bool-conversion]
         int R__n=(&R__stl) ? int(R__stl.size()) : 0;
                    ^~~~~~  ~
/Users/laurent/o2/o3work/sw/BUILD/960d7f0f83e6fe29408f15ac622a951e491697ff/FairRoot/base/G__BaseDict.cxx:7006:21: warning: reference cannot be bound to dereferenced null pointer in well-defined C++ code; pointer may be assumed to always convert to true [-Wundefined-bool-conversion]
         int R__n=(&R__stl) ? int(R__stl.size()) : 0;
                    ^~~~~~  ~
4 warnings generated.
[ 56%] Linking CXX shared library ../lib/libBase.so
[ 56%] Built target Base
make: *** [all] Error 2
17:09:19 laurent@nanpc420:~/o2/o3work>

[ktf]

Ok, I managed to reproduce the problem. It's actually in sodium (which is required by zeromq), not FairRoot. For the moment you can work it around either by installing zeromq with home-brew or by doing git clone https://github.com/jedisct1/libsodium at the same level of alidist. I am working on a proper fix.

[aphecetche]

I went the 'brew install zeromq' route... Now failing later on in O2.

see the sw/BUILD/O2-latest/log file attached

log.txt

[ktf]

Do you have ROOTSYS explicitly set on your system?

[ktf]

Do you still have problems with this?

[aphecetche]

Hi,

Sorry for the delay.

Tried again "from scratch", following : http://alisw.github.io/o2-tutorial/#!/1/4 (but bypassing the local copy of O2 altogether for this test)

git clone https://github.com/alisw/alibuild │
git clone -b IB/v5-06/next https://github.com/alisw/alidist
alibuild/aliBuild build O2

and I get a problem in DDS :

cp: /Users/laurent/o2/o3work/sw/BUILD/243487caf723c530fbc6e7173526e664f76ddfaf/DDS/dds-wrk-│
bin/libboost_log-mt.dylib: Permission denied │
ERROR: Can not build DDS worker package. Missing dependency.

[ktf]

Mmm.. the branch name should have been IB/v5-08/next by now. Can you check with that?

[aphecetche]

OK. It's getting worse ;-)

From a fresh directory :

git clone https://github.com/alisw/alibuild
git clone -b IB/v5-08/next https://github.com/alisw/alidist
alibuild/aliBuild build O2

ERROR: Error while executing /Users/laurent/o2/o4work/sw/SPECS/osx_x86-64/Python-modules/1.0-1/build.sh on `nanpc420.local'.
ERROR: Log can be found in /Users/laurent/o2/o4work/sw/BUILD/Python-modules-latest/log

(that log is attached)

log.zip

[ktf]

You are now picking up python from the system, but for some reason it does not find Python.h...
Can you tell me what:

python --version && which python

tells you? Are you using some special setup for python?

[aphecetche]

python --version && which python
Python 2.7.11
/usr/local/bin/python

I'm not a "pythonist" so I don't setup anything special for python myself. But could it be that some other things messed with my Python installation ? How can I check / what should I check ?

[ktf]

Indeed looks like you have a non system installation of python which has the header in some weird place. I will try to see if I can find a workaround.

[ktf]

Can you tell me what:

from distutils.sysconfig import get_python_inc
get_python_inc()

tells you?

[aphecetche]

Hi,

Any news on this one ? I'm pretty much stuck by it...

[ktf]

I assume this is not an issue anymore? Please reopen if not the case.