sunarditay/rl_local_planner

Mobile-robot Collision Avoidance Learning

Overview

---

• To navigation on the complex environment, the Deep Reinforcement Learning can be attractive method better than classical methods. rl_local_planner inherits MRCA(Multi-robot collision avidance) methodology, which is based on PPO(Proximal Policy Optimization and Stage simulation. In our case, we adopt SAC as state-of-art Model-Free algorithm and trained on the physical simulation to be considered with real world properties. The training time are decreased by replacing with off-policy and entropy exploration. The off-policy could be more sample efficient because of multi agents environment.
• The rl_local_planner is already tested on real logistic robot and ported the ROS but not local planner planner. For practical usages, rl_local_planner included the global_planner and selected the sub goal concept as look ahead, called MCAL(Mobile robot Collision Avoidance Learning) similar with Hybrid-MRCA.

Requirement

---

• python 2.7
• ROS Kinetic & Melodic
• mpi4py
• Stage
• Gazebo
• PyTorch

Mobile-robot Collision Avoidance Learning

---

Stage world

---

How to Train

---

Please use the stage_ros-add_pose_and_crash package instead of the default package provided by ROS.

mkdir -p catkin_ws/src
cd catkin_ws/src
git clone https://github.com/CzJaewan/rl_local_planner.git
cd ..
catkin_make

To train, modify the hyper-parameters in sac_main.py as you like, and running the following command:

cd catkin_ws/src/rl_local_planner/stage_ro-add_pose_and_crach/
rosrun stage_ros_add_pose_and_crash stageros_w worlds/stage1.world
mpiexec -np 6 python sac_main.py

How to Test

---

rosrun stage_ros_add_pose_and_crash stageros worlds/circle.world
mpiexec -np 50 python circle_test.py

Gazebo world

---

How to Train

---

Please use the stage_ros-add_pose_and_crash package instead of the default package provided by ROS.

mkdir -p catkin_ws/src
cp stage_ros-add_pose_and_crash catkin_ws/src
cd catkin_ws
catkin_make
source devel/setup.bash

To train, modify the hyper-parameters in sac_main.py as you like, and running the following command:

rosrun stage_ros_add_pose_and_crash stageros worlds/stage1.world
mpiexec -np 6 python sac_main.py

How to Test

---

1. Run Gazebo simulator

#empty world
roslaunch sp_gazebo scan_merged_sr7e_rl_gazebo.launch

#obstacle world
roslaunch sp_gazebo scan_merged_sr7e_rl_static_2_gazebo.launch
roslaunch sp_gazebo scan_merged_sr7e_rl_static_4_gazebo.launch
roslaunch sp_gazebo scan_merged_sr7e_rl_d35m_gazebo.launch
roslaunch sp_gazebo scan_merged_sr7e_rl_dc_gazebo.launch
roslaunch sp_gazebo scan_merged_sr7e_rl_df_gazebo.launch

2. Run RL navigation

without Look ahead (mcal)

1. modify import world file of gazebo_test_sac.py

   cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/single
   gedit gazebo_test_sac.py
   ****************************************************
   from syscon_gazebo_test_amcl_world import StageWorld

2. run roslaunch file & runfile

   roslaunch gazebo_rl_test sysconbot_rl.launch
   cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/single
   mpiexec -np 1 python gazebo_test_sac.py

with Look ahead (mcal_p)

1. modify import world file of gazebo_test_sac.py

   cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/single
   gedit gazebo_test_sac.py
   ****************************************************
   from syscon_gazebo_test_amcl_world_LAH import StageWorld

2. run roslaunch file & runfile

   roslaunch gazebo_rl_test sysconbot_rl_lookahead_2.launch
   cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/single
   mpiexec -np 1 python gazebo_test_sac.py

with Look ahead & hybrid mode (hybrid mcal_p)

roslaunch gazebo_rl_test hybrid_syscon_navigation.launch
cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/single
mpiexec -np 1 python gazebo_test_sac.py

1. modify import world file of gazebo_test_sac.py

   cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/single
   gedit gazebo_test_sac.py
   ****************************************************
   from syscon_gazebo_test_amcl_world_LAH_hybrid import StageWorld

2. run cmd_vel_swich node

   cd ~/catkin_ws/src/rl_local_planner/navigation/script
   python hybrid_cmd_vel_swich.py

3. run roslaunch file & runfile

   roslaunch gazebo_rl_test hybrid_syscon_navigation.launch
   cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/single
   mpiexec -np 1 python gazebo_test_sac.py

How to Test (multi-agent)

---

1. Run Gazebo simulator

# 2 agent
roslaunch sp_gazebo scan_merged_sr7e_rl_2agent.launch

# 4 agent
roslaunch sp_gazebo scan_merged_sr7e_rl_4agent.launch

2. Run RL navigation

localization, global planner, costmap, map run

# 2 agent
roslaunch gazebo_rl_test multi-sysconbot_rl_lookahead_2.launch

# 4 agent
roslaunch gazebo_rl_test multi-sysconbot_rl_lookahead_4.launchy

rl based collision avoidance and driving controller run

# 2 agent
cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/multi
mpiexec -np 2 python gazebo_1_multi_test_sac.py
or
mpiexec -np 1 python gazebo_1_multi_test_sac.py
mpiexec -np 1 python gazebo_2_multi_test_sac.py

# 4 agent
cd ~/catkin_ws/src/rl_local_planner/mcal_gazebo/GAZEBO_TEST_SAC/multi
mpiexec -np 4 python gazebo_1_multi_test_sac.py
or
mpiexec -np 1 python gazebo_1_multi_test_sac.py
mpiexec -np 1 python gazebo_2_multi_test_sac.py
mpiexec -np 1 python gazebo_3_multi_test_sac.py
mpiexec -np 1 python gazebo_4_multi_test_sac.py

Real world test

References:

A study on dynamic object avoidance driving based on Reinforcement learning using mobile robot