spyroscvrs/robotanticipationofhumanactions

Robot Anticipation Of Human Actions Using an RGB-D sensor

The project is aimed to allow a robot to anticipate human intentions while interaction partners are still performing their action. Specifically, two actions are anticipated from the UR5 robot. When the human decides for an object to pick and reaches to it, the robot arm must pick the other object. This way, a feeling of cooperation between the interaction partner and the robot will be created.

Getting Started

One Ubuntu 16.04 LTS machine and a Windows 10 machine is needed.

Prerequisites

• ROS-kinetic: http://wiki.ros.org/kinetic/Installation/Ubuntu
  
• Catkin workspace: http://wiki.ros.org/ROS/Tutorials/InstallingandConfiguringROSEnvironment
  
• ROS Industrial Universal Robots: https://github.com/ros-industrial/universal_robot
  
• Visual Studio 2017: https://docs.microsoft.com/en-us/visualstudio/ide/?view=vs-2017
  
• Xbox Kinect v2 sensor
  
• Kinect to PC adaptor
  

SETUP

1. Get ROS kinetic on Linux 16.04
   

2. Follow instructions to create catkin workspace
   

3. Install ROS Industrial Universal Robots
   

Open terminal and execute:

sudo apt-get install ros-kinetic-universal-robot

4. Download files from my GIT repository
   

File 'listener.py' is needed into the Ubuntu machine inside catkin enviroment, into /src folder.

5. Download Visual Studio 2017 on Windows Machine
   

6. Connect Xbox Kinect v2 to the Windows Machine using an adaptor
   

7. On Ubuntu Machine open terminal and run
   

hostname -I

8. Go to BodyBasics-WPF folder on Windows
   

Certain changes are needed to use the application.
Open MainWindow.xaml.cs using Visual studio or any other text editor.
Go to line 200, change the IP, to the IP of the Ubuntu Machine obtained in step 7.

TESTING

The application is ready to be simulated using Gazebo simulator.
On Ubuntu Machine side:
Make sure for every terminal/shell you are directed to the catkin enviroment.
Example:

cd catkin_ws

and then source the setup file for your shell using the command:

source devel/setup.bash

1. Open terminal and run Gazebo simulation

roslaunch ur_gazebo ur5.launch

2. Open new terminal for setting up the MoveIt! nodes to allow motion planning

roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch sim:=true

3. Open new terminal for starting up RViz with a configuration

roslaunch ur5_moveit_config moveit_rviz.launch config:=true

4. Open new terminal to run listener node

cd src

python listener.py

On Windows Machine side:

Open the BodyBasics-WPF folder and then open the BodyBasics-WPF.sln application using Visual Studio 2017 and click start.

Photos

The photos were obtained in the Advanced Robotics at Queen Mary using the UR5 robot

Video

The video was obtained in the Advanced Robotics at Queen Mary using the UR5 robot

https://www.youtube.com/watch?v=aitVCOc7kYU

Author

Spyridon Couvaras
Email: [email protected]

Acknowledgments

• Dr Lorenzo Jamone
• Mr Gokhan Solak
• Mr Brice Denoun