snoplus/rat-container

A Docker recipe for building a SNO+ environment for RAT.

rat-container's Issues

Graphics not working in Ubuntu 20

When using the latest rat-container in Ubuntu 20 using the command docker run -ti --rm --user $(id -u) -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v /home/jonathan/sno+/rat/:/rat jamierajewski/rat-container, the graphical output (such as TBrowser in ROOT) does not work. The error message is:

Error in <TGClient::TGClient>: can't open display ":0", switching to batch mode...
 In case you run from a remote ssh session, reconnect with ssh -Y

However, using the remote shifting container with the command docker run -ti --net=host --rm --user $(id -u) -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix registry.gitlab.com/jamierajewski/remote-shift-container/release graphical output works with no problems.

Create ROOT 6 version

Not completely necessary, but if it isn't much work it could be useful for those who would rather use that.

Add example of running this on the cluster

As many users want to run jobs on the cluster with this container, an example submission script for this will be beneficial.

Add X11 forwarding or some method of using GUIs through the Docker version

Docker needs some kind of pass-through in order for GUIs to be used, whereas Singularity does not. This is very important as some of the tools (like ROOT) have GUI tools.

Add best-practice features

This repo contains some simple best-practices for building Docker containers, primarily focused around security but also stability.

• Since I have a custom entrypoint already, add --init option to all run commands in the docs which will use tini
• Add nonroot user as specified
• No need for bind-tools, and not using alpine linux

The nonroot user creation may require slightly modified commands since this container is CentOS-based while that one is Alpine-based.

Test and change docs for Windows and macOS to use localhost

Rather than have the user find their own IP and map it in, there is a docker feature which allows for the mapping of localhost.

Example for macOS:

xhost + 127.0.0.1

docker run -e DISPLAY=host.docker.internal:0 my-container

Add tools to build the RAT docs

Latex2html is required to build the docs, so if it isn't too large add it to the image.

Error for command "run": unknown flag: --app

Reported by a user when running the step to build RAT ( --app build-rat)

Could not find /rat/env.sh

I am using Docker for Windows 10, and I get the following errors when I try to source:

The first suggests that I can ignore the messages because I'm building RAT, but the second seems to be a failure ("bad interpreter: No such file or directory").
I tried two times with forward slashes instead of backslashes.

I suspect the problem is with my RAT repo: I cloned from here, which doesn't have a scripts folder as "source /home/scripts/setup-env.sh" might suggest it should.
Am I doing something wrong, and if so, what is it?
Note that I changed the directory name from rat-pac to rat to be congruent with this readme.

geant4, scons.

are these issues or am i doing something wrong? @jamierajewski

root6 container:
/home/scripts/setup-env.py contains:
source /home/software/geant4.10.00.p04
but in this container geant4.xxxxxxx.p02 is installed

root5 container:
in /home/software there is no scons

Upgrade ROOT to 5.34.38

Following internal discussion, there is a minor upgrade to the ROOT version being used for bugfixes and additional functionality.

Should simply have to change the version here:

rat-container/Dockerfile

Line 26 in 4c73690

tar zxvf root_v5.34.36.source.tar.gz && \

Add TensorFlow/cppflow

There are ongoing tests using these tools for fitting, so adding them to the image will be beneficial

Add CI/CD to deploy new versions to cvmfs automatically

At the moment, when a there are new changes made to master, Dockerhub and SingularityHub will automatically rebuild the image so that they always host the latest; however, the new image will not make its way to cvmfs unless someone remembers to do so.

Some kind of continuous integration/deployment technique could be used so that a new change on master will trigger a rebuild of the latest image and then a pull to cvmfs.

Find better solution than sourcing setup scripts

Whenever you wish to use the container, you must source the setup-env.sh script (and the build-rat.sh script if building). This can be annoying, and there should be a way to manage what they do directly on build time in Docker/Singularity; however, I had issues getting the ENV command to persist when setting up the environment in the container so more research is necessary.

Singularity version doesn't work anymore

This was pointed out after fixing #14 which made it easier to use the Docker version, but broke the Singularity one; this is because entrypoints don't translate to Singularity, so the environment setup never occurs.

A simple solution is to still include the setup-env.sh script, and restore those instructions for Singularity only.

Create Docker version

Because the container is ultimately not that useful on clusters anyways due to the tools being installed under CVMFS, creating a Docker version would be significantly easier to use on host machines.

By default, Docker allows admin privileges inside the container, so being able to edit the contents is natively supported. Not to mention Docker is much easier to install on every OS (whereas Singularity is cumbersome in that respect).

Create run scripts for each of the OS's

It is incredibly cumbersome to have to either type out of copy-paste the entire docker GUI commands, so create a run script that will take in the path to RAT and any other bind-mounts, and then apply them to the docker command.

This can be done now that I am aware of the host.docker.internal networking feature, which should allow for easier IP mapping to localhost rather than have the user find their own IP and map it.

Create 'base' docker image and base the different versions on it

Having a base image makes management of new libs and tools much easier as changes don't have to be copy-pasted between the Dockerfiles; simply have the base be everything up to where the ROOT version is chosen (as that is where it splits into the ROOT5 and 6 versions)

Trouble on OSX

So, I tried to install RAT using the container (clean build) both using singularity and docker and in both cases hit the same problem. I strictly followed the instructions outlined in the README.md, but once I proceed to the build phase I get:

root@f2952470271d:/rat# scons
scons: Reading SConscript files ...

  Performing checks to build Postgres library...

Determining size align of short...Test failed.
root@f2952470271d:/rat#

(the error in singularity is the same).

Looking at the config.log in the RAT directory I get the following:

root@f2952470271d:/rat# cat config.log
file /rat/config/PSQL.scons,line 626:
	Configure(confdir = .sconf_temp)
scons: Configure: Determining size align of short...
scons: Configure: Caught exception while building ".sconf_temp/conftest_0.c":

Traceback (most recent call last):
  File "/home/software/scons-3.1.2/script/../engine/SCons/Job.py", line 199, in start
    task.prepare()
  File "/home/software/scons-3.1.2/script/../engine/SCons/Taskmaster.py", line 179, in prepare
    self.exception_raise()
  File "/home/software/scons-3.1.2/script/../engine/SCons/Taskmaster.py", line 559, in _exception_raise
    exec("raise exc_type, exc_value, exc_traceback")
  File "/home/software/scons-3.1.2/script/../engine/SCons/Taskmaster.py", line 984, in next_task
    task.make_ready()
  File "/home/software/scons-3.1.2/script/../engine/SCons/Taskmaster.py", line 411, in make_ready_current
    (not t.always_build and t.is_up_to_date())
  File "/home/software/scons-3.1.2/script/../engine/SCons/Node/FS.py", line 3528, in is_up_to_date
    r = self.changed()
  File "/home/software/scons-3.1.2/script/../engine/SCons/Node/FS.py", line 3280, in changed
    has_changed = SCons.Node.Node.changed(self, node)
  File "/home/software/scons-3.1.2/script/../engine/SCons/Node/__init__.py", line 1477, in changed
    bi = node.get_stored_info().binfo
  File "/home/software/scons-3.1.2/script/../engine/SCons/Node/FS.py", line 2889, in get_stored_info
    sconsign_entry = self.dir.sconsign().get_entry(self.name)
  File "/home/software/scons-3.1.2/script/../engine/SCons/Node/FS.py", line 1894, in sconsign
    return _sconsign_map[self._func_sconsign](self)
  File "/home/software/scons-3.1.2/script/../engine/SCons/Node/FS.py", line 74, in sconsign_dir
    node._sconsign = SCons.SConsign.ForDirectory(node)
  File "/home/software/scons-3.1.2/script/../engine/SCons/SConsign.py", line 235, in __init__
    db, mode = Get_DataBase(dir)
  File "/home/software/scons-3.1.2/script/../engine/SCons/SConsign.py", line 89, in Get_DataBase
    db = DataBase[top] = DB_Module.open(DB_Name, "c")
  File "/home/software/scons-3.1.2/script/../engine/SCons/dblite.py", line 210, in open
    return dblite(file, flag, mode)
  File "/home/software/scons-3.1.2/script/../engine/SCons/dblite.py", line 129, in __init__
    self._dict = pickle.loads(p)
ValueError: unsupported pickle protocol: 4
root@f2952470271d:/rat#

It seems that the failure is due to an exception from python. I call your attention to the second to last line in the log above. Checking over the internet, it seems that version 4 of the pickle protocol was introduced in python 3.4. However, the default python version in the container is 2.7.6:

root@f2952470271d:/# python --version
Python 2.7.6

My suspicion is that scons was built with python 3.4, but when being called it loads python 2.7. Now, why I am the only one seeing this error is a mystery.

Any suggestions? If it helps, I am running OSX Mojave. But this should not be related with the host system.