osrf/ariac-docker

Containerization of ARIAC competition server and competitors' code. See https://bitbucket.org/osrf/ariac/wiki/Home

ariac-docker's Issues

dockerhub images assume user id 1000

The dockerfiles have been written to take user id into account, but now that we ship docker images instead of have users build them themselves, the user id needs to be taken into account at run time.

Support batch running of multiple competition trials

Support use without nvidia-docker

Gazebo missing crash logs

Note: After some fiddling with the run commands, I got everything to work. It was due to no display available (obviously) However there is no error / log output that indicates this.

Error 1: Gazebo / gear crashes

... logging to /home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/roslaunch-06fab0b7b083-322.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://172.18.0.22:44625/

SUMMARY
========

PARAMETERS
 * /rosdistro: melodic
 * /rosversion: 1.14.3

NODES
  /
    ariac_sim (osrf_gear/gear.py)

auto-starting new master
process[master]: started with pid [332]
ROS_MASTER_URI=http://172.18.0.22:11311

setting /run_id to e5501c2e-3fc2-11ea-b2fb-0242ac120016
process[rosout-1]: started with pid [343]
started core service [/rosout]
writing file /tmp/ariac/arm1.urdf.xacro
writing file /tmp/ariac/arm2.urdf.xacro[350]
writing file /tmp/ariac/gear.world
writing file /tmp/ariac/gear.launch
writing file /tmp/ariac/gear.urdf.xacro
Running command: roslaunch /tmp/ariac/gear.launch world_path:=/tmp/ariac/gear.world gear_urdf_xacro:=/tmp/ariac/gear.urdf.xacro arm_urdf_dir:=/tmp/ariac/
... logging to /home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/roslaunch-06fab0b7b083-353.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

xacro: in-order processing became default in ROS Melodic. You can drop the option.
xacro: in-order processing became default in ROS Melodic. You can drop the option.
xacro: in-order processing became default in ROS Melodic. You can drop the option.
started roslaunch server http://172.18.0.22:43853/

SUMMARY
========

PARAMETERS
 * /ariac/arm1/arm/action_monitor_rate: 10
[Model data loads successfully] 
 * /sensors_tf_publisher/tf_prefix: 
 * /use_sim_time: True

NODES
  /ariac/arm2/
    arm_controller_spawner (controller_manager/controller_manager)
    fake_joint_calibration (rostopic/rostopic)
    joint_state_controller_spawner (controller_manager/controller_manager)
    robot_state_publisher (robot_state_publisher/robot_state_publisher)
    spawn_gazebo_model (gazebo_ros/spawn_model)
    tf_relay (osrf_gear/tf2_relay)
  /ariac/arm1/
    arm_controller_spawner (controller_manager/controller_manager)
    fake_joint_calibration (rostopic/rostopic)
    joint_state_controller_spawner (controller_manager/controller_manager)
    robot_state_publisher (robot_state_publisher/robot_state_publisher)
    spawn_gazebo_model (gazebo_ros/spawn_model)
    tf_relay (osrf_gear/tf2_relay)
  /
    gazebo (gazebo_ros/gzserver)
    gazebo_gui (gazebo_ros/gzclient)
    gazebo_unpauser (osrf_gear/gazebo_unpauser.py)
    sensors_tf_publisher (robot_state_publisher/robot_state_publisher)
    spawn_gazebo_sensors (gazebo_ros/spawn_model)
    startup_ariac (osrf_gear/startup.sh)

ROS_MASTER_URI=http://172.18.0.22:11311

process[startup_ariac-1]: started with pid [377]
Ensuring scoring log file exists
File already exists: /home/ariac/.ariac/log/performance.log
[startup_ariac-1] process has finished cleanly
log file: /home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/startup_ariac-1*.log
process[gazebo-2]: started with pid [379]
[ INFO] [1579991693.966447553]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1579991693.968303032]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...
process[gazebo_gui-3]: started with pid [384]
[ INFO] [1579991694.429378132]: Finished loading Gazebo ROS API Plugin.
process[sensors_tf_publisher-4]: started with pid [450]
process[spawn_gazebo_sensors-5]: started with pid [514]
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
  what():  boost: mutex lock failed in pthread_mutex_lock: Invalid argument
Aborted (core dumped)
================================================================================REQUIRED process [gazebo_gui-3] has died!
process has died [pid 384, exit code 134, cmd /opt/ros/melodic/lib/gazebo_ros/gzclient __name:=gazebo_gui __log:=/home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/gazebo_gui-3.log].
log file: /home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/gazebo_gui-3*.log
Initiating shutdown!
================================================================================
[ariac/arm2/tf_relay-6] killing on exit
[spawn_gazebo_sensors-5] killing on exit
[sensors_tf_publisher-4] killing on exit
Traceback (most recent call last):
  File "/opt/ros/melodic/lib/gazebo_ros/spawn_model", line 31, in <module>
    from tf.transformations import quaternion_from_euler
  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf/__init__.py", line 30, in <module>
    from tf2_ros import TransformException as Exception, ConnectivityException, LookupException, ExtrapolationException
  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf2_ros/__init__.py", line 39, in <module>
    from .buffer_interface import *
  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf2_ros/buffer_interface.py", line 32, in <module>
    import roslib; roslib.load_manifest('tf2_ros')
  File "/opt/ros/melodic/lib/python2.7/dist-packages/roslib/launcher.py", line 62, in load_manifest
    sys.path = _generate_python_path(package_name, _rospack) + sys.path
  File "/opt/ros/melodic/lib/python2.7/dist-packages/roslib/launcher.py", line 93, in _generate_python_path
    m = rospack.get_manifest(pkg)
  File "/usr/lib/python2.7/dist-packages/rospkg/rospack.py", line 167, in get_manifest
    return self._load_manifest(name)
  File "/usr/lib/python2.7/dist-packages/rospkg/rospack.py", line 211, in _load_manifest
    retval = self._manifests[name] = parse_manifest_file(self.get_path(name), self._manifest_name, rospack=self)
  File "/usr/lib/python2.7/dist-packages/rospkg/manifest.py", line 410, in parse_manifest_file
    from rosdep2.rospack import init_rospack_interface, is_ros_package, is_system_dependency, is_view_empty
  File "/usr/lib/python2.7/dist-packages/rosdep2/__init__.py", line 45, in <module>
    from .lookup import RosdepDefinition, RosdepView, RosdepLookup, \
  File "/usr/lib/python2.7/dist-packages/rosdep2/lookup.py", line 44, in <module>
    from .sources_list import SourcesListLoader
  File "/usr/lib/python2.7/dist-packages/rosdep2/sources_list.py", line 48, in <module>
    from .gbpdistro_support import get_gbprepo_as_rosdep_data, download_gbpdistro_as_rosdep_data
  File "/usr/lib/python2.7/dist-packages/rosdep2/gbpdistro_support.py", line 18, in <module>
    from .platforms.debian import APT_INSTALLER
  File "/usr/lib/python2.7/dist-packages/rosdep2/platforms/debian.py", line 36, in <module>
    from .pip import PIP_INSTALLER
  File "/usr/lib/python2.7/dist-packages/rosdep2/platforms/pip.py", line 33, in <module>
    import pkg_resources
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3088, in <module>
    @_call_aside
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3072, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3124, in _initialize_master_working_set
    list(map(working_set.add_entry, sys.path))
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 614, in add_entry
[gazebo_gui-3] killing on exit
    for dist in find_distributions(entry, True):
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1964, in find_on_path
    for dist in factory(fullpath):
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2034, in distributions_from_metadata
[gazebo-2] killing on exit
    root, entry, metadata, precedence=DEVELOP_DIST,
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2473, in from_location
    py_version=py_version, platform=platform, **kw
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2835, in _reload_version
    md_version = _version_from_file(self._get_metadata(self.PKG_INFO))
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2437, in _version_from_file
    line = next(iter(version_lines), '')
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2626, in _get_metadata
    if self.has_metadata(name):
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2666, in __getattr__
    def __getattr__(self, attr):
KeyboardInterrupt
process[ariac/arm2/tf_relay-6]: started with pid [550]
RLException: cannot add process [ariac/arm2/spawn_gazebo_model-7] after process monitor has been shut down
The traceback for the exception was written to the log file
================================================================================REQUIRED process [ariac_sim-2] has died!
process has died [pid 350, exit code 1, cmd /opt/ros/melodic/lib/osrf_gear/gear.py --development-mode --visualize-sensor-views -f /opt/ros/melodic/share/osrf_gear/config/sample.yaml /opt/ros/melodic/share/osrf_gear/config/sample_user_config.yaml __name:=ariac_sim __log:=/home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/ariac_sim-2.log].
log file: /home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/ariac_sim-2*.log
Initiating shutdown!
================================================================================
[ariac_sim-2] killing on exit
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

Error 2: Says there are logs but there are none
There are a few log locations:
/home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/gazebo_gui-3*.log
/home/ariac/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016/ariac_sim-2*.log
However if I cd to each:

ariac@06fab0b7b083:~/.ros/log/e5501c2e-3fc2-11ea-b2fb-0242ac120016$ ls
ariac-arm2-tf_relay-6-stdout.log  master.log  roslaunch-06fab0b7b083-322.log  roslaunch-06fab0b7b083-353.log  rosout-1-stdout.log  rosout.log  sensors_tf_publisher-4-stdout.log

Reproduce

/bin/bash $HOME/ariac-docker/ariac-competitor/ariac_network.bash
/bin/bash $HOME/ariac-docker/ariac-server/build-images.sh

docker run --rm -it --name test -e XAUTHORITY=/tmp/.docker.xauth \
 -e ROS_IP=172.18.0.22 -e ROS_MASTER_URI=http://172.18.0.22:11311 \
--ip 172.18.0.22 --net ariac-network -v /etc/localtime:/etc/localtime:ro \
-v /tmp/.docker.xauth:/tmp/.docker.xauth -v /dev/log:/dev/log \
-v logs:/home/cloudsim/gazebo-logs ariac-server-melodic /bin/bash

Then, attempt

roslaunch osrf_gear sample_environment.launch

per: wiki/2019/tutorials/gear_interface

Personal Side
I just want to be able to start the ARIAC warehouse environment and have the gazebo GUI show up. Possibly interactively send commands to see how everything behaves. I understand that there is a script for capturing video, but I want something realtime. My primary concern is that the logs are missing, so I have no idea what gazebo is expecting from me.

TD:DR
I start the docker file after setting up the ariac_network.bash, build-images.sh, and finally the final command in run_container.bash with interactivity turned on so that I have access to the bash terminal. Executing a basic osrf_gear launch file crashes without any log files. The crash is related to the missing display of which I have solved

Can't open display. Also - Gazebo 7.7 or 8.3 for qualifier?

I just ran through the tutorial on the master branch and saw that Gazebo 7.7 is being used. I saw @dhood mentioned in PR#17 that 7.7 will be used until logging issues in 8.3 are fixed. Any idea if this will happen before the March 2018 qualifier?

Also, should it be opening a display?

[Err] [RenderEngine.cc:730] Can't open display: 
[Wrn] [RenderEngine.cc:97] Unable to create X window. Rendering will be disabled

I saw in PR #6 that GPU support was removed, but I'm not sure if it should be opening an X window anyway. I see the TODO about re-enabling nvidia support in run_container.bash

Don't run user code as root

Push Docker images with ariac2.0.4 to dockerhub

Users playing back logs recorded with 2.0.3 on host machines with 2.0.4 installed will not see the conveyor belt in the workcell.

Network Error on ROS Kinetic - Cannot preparing team's system

Looks like there is some network issue with preparing workspace for ROS kinetic, cannot fetch anything from the Internet.

Host system: Ubuntu 16.04 + ROS Kinetic
Using team config: link

Error log:

Step 4/9 : RUN /build_team_system.bash
 ---> Running in fc1180552e36
Preparing environmen for running cwru_ariac
Err:1 http://packages.osrfoundation.org/gazebo/ubuntu-stable xenial InRelease
  Temporary failure resolving 'packages.osrfoundation.org'
Err:2 http://archive.ubuntu.com/ubuntu xenial InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://packages.ros.org/ros/ubuntu xenial InRelease
  Temporary failure resolving 'packages.ros.org'
Err:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Err:5 http://archive.ubuntu.com/ubuntu xenial-security InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-updates/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-security/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://packages.osrfoundation.org/gazebo/ubuntu-stable/dists/xenial/InRelease  Temporary failure resolving 'packages.osrfoundation.org'
W: Failed to fetch http://packages.ros.org/ros/ubuntu/dists/xenial/InRelease  Temporary failure resolving 'packages.ros.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package ros-kinetic-trac-ik*
E: Couldn't find any package by glob 'ros-kinetic-trac-ik*'
E: Couldn't find any package by regex 'ros-kinetic-trac-ik*'
Cloning into 'cwru_ariac'...
fatal: unable to access 'https://github.com/cwru-robotics/cwru_ariac.git/': Could not resolve host: github.com
Cloning into 'catkin_simple'...
fatal: unable to access 'https://github.com/catkin/catkin_simple.git/': Could not resolve host: github.com

ROS Ubuntu Bionic: public key is not available

Problem

I am running on Ubuntu 18.04, and my docker build crashes at:

# setup keys and sources for official Gazebo and ROS debian packages
RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys D2486D2DD83DB69272AFE98867170598AF249743 \
 && echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable bionic main" > /etc/apt/sources.list.d/gazebo-latest.list \
 && apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 421C365BD9FF1F717815A3895523BAEEB01FA116 \
 && echo "deb http://packages.ros.org/ros/ubuntu bionic main" > /etc/apt/sources.list.d/ros-latest.list

I get the message:

Step 5/27 : RUN apt-get update && apt-get install -q -y     ariac3     bash-completion     gazebo9     libgazebo9-dev     locales     psmisc     python-rosdep     python-rosinstall     python-vcstools     ros-melodic-robot-state-publisher     ros-melodic-ros-controllers     ros-melodic-ros-core     ros-melodic-ros-base     wget     && apt-get clean     && rm -rf /var/lib/apt/lists/*
 ---> Running in 7a8256d8841c
Get:1 http://packages.ros.org/ros/ubuntu bionic InRelease [4669 B]
Get:2 http://packages.osrfoundation.org/gazebo/ubuntu-stable bionic InRelease [4253 B]
Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Err:1 http://packages.ros.org/ros/ubuntu bionic InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F42ED6FBAB17C654
Get:4 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
...
Reading package lists...
W: GPG error: http://packages.ros.org/ros/ubuntu bionic InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F42ED6FBAB17C654
E: The repository 'http://packages.ros.org/ros/ubuntu bionic InRelease' is not signed.

Solution
I fixed this via:

# setup keys and sources for official Gazebo and ROS debian packages
RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys D2486D2DD83DB69272AFE98867170598AF249743 \
 && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F42ED6FBAB17C654 \
 && echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable bionic main" > /etc/apt/sources.list.d/gazebo-latest.list \
 && apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 421C365BD9FF1F717815A3895523BAEEB01FA116 \
 && echo "deb http://packages.ros.org/ros/ubuntu bionic main" > /etc/apt/sources.list.d/ros-latest.list

Let me know if doing a PR is a good idea. Not sure how many people had this issue, but my computer was pretty vanilla (1804, basic Docker version 19.03.5, build 633a0ea838).

Support kinetic in addition to indigo