ethz-asl/navrep

New 24/11/20: we compared the Guldenring et al. pretrained models (IROS 2020), see here.

NavRep

This is the official repository for the NavRep paper. it contains:

• a simulator aimed at allowing anyone to easy reproduce and improve on the state-of-the-art of RL for robot navigation.
• a suite for unsupervised-learning-assisted RL for robot navigation. It contains tools, datasets, models which allow you to easily reproduce findings from our paper.

Publication

Daniel Dugas, Juan Nieto, Roland Siegwart and Jen Jen Chung, NavRep: Unsupervised Representations for Reinforcement Learning of Robot Navigation in Dynamic Human Environments, Pending review, 2020 - pdf

Pre-requisites

Python 3.6

For example, on Ubuntu 20

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install -y python3.6 python3.6-dev
sudo apt-get install -y virtualenv python3-virtualenv

We recommend creating a virtualenv:

virtualenv ~/navrepvenv --python=python3.6
source ~/navrepvenv/bin/activate

rospy:

pip install --extra-index-url https://rospypi.github.io/simple/ rospy rosbag tf tf2_ros

specific versions of Keras, gym (later versions cause errors with some user-facing scripts)

pip install gym==0.15.7 keras==2.3.1

Install

Installing navrep is as simple as

pip install navrep

Try it!

python -m navrep.envs.navreptrainenv

Press w, a, s, d to move around, shift to speed up the simulation.

NavRepSim

NavRep training environment

NavRep testing environment

SOADRL

CrowdMove

Pfeiffer et al.

Reproducing NavRep

The following is an example of how to reproduce the results in the NavRep paper. It is a subset of the full procedure, which can be found in the workflow_navrep.sh file.

make a V dataset

python -m navrep.scripts.make_vae_dataset --environment navreptrain --render

(remove --render argument to speed up data generation)

train V

python -m navrep.scripts.train_vae --environment navreptrain

train M

python -m navrep.scripts.make_rnn_dataset --environment navreptrain
python -m navrep.scripts.train_rnn --environment navreptrain

Play encoded environment

python -m navrep.scripts.play_navreptrainencodedenv --backend VAE_LSTM --encoding V_ONLY

Play dream environment

Train C

python -m navrep.scripts.train_gym_navreptrainencodedenv --backend VAE_LSTM --encoding V_ONLY

Test C

python -m navrep.scripts.cross_test_navreptrain_in_ianenv --backend VAE_LSTM --encoding V_ONLY --render

Run in ROS node

roslaunch launch/sim_and_navrep.launch

Using NavRep with your own models / environments

• Testing your model in the NavRep test environment

Pre-trained NavRep models

all models shown in the paper are available here.

Copy the models from the V, M (modular archs) W (joint and transformer archs) and gym (C module) folders to your home directory according to the following structure:

~/navrep/
  └── models
      ├── gym
      ├── M
      └── V

run the following, and the models inside the ~/navrep directory will be tested in the navrep test environment:

python -m navrep.scripts.cross_test_navreptrain_in_ianenv --backend VAE_LSTM --encoding V_ONLY --render

flags:

--backend

• VAE1D_LSTM (Modular V, M with 1D lidar representation)
• VAE_LSTM (Modular V, M with rings lidar representation)
• VAE1DLSTM (Joint V+M with 1D lidar representation)
• VAELSTM (Joint V+M with rings lidar representation)
• GPT1D (Transformer with 1D lidar representation)
• GPT (Transformer with rings lidar representation)
• E2E1D (End-to-end with 1D lidar representation)
• E2E (End-to-end with rings lidar representation)

--encoding

• V_ONLY (a.k.a only z features)
• M_ONLY (a.k.a only h features)
• VM (a.k.a z+h features)
• VCARCH (for End-to-end only, specifies convolutional architecture)
• CARCH (for End-to-end only, specifies small architecture)

Tested NavRep training regimes

Environments	Modular V, M	Joint V+M	Transformer V+M
NavRep train	✔️	✔️	✔️
NavRep test	✔️	✔️	✔️
SOADRL	✔️	✔️	✔️
Pfeiffer	✔️	✔️	✔️
CrowdMove	➖	➖	➖

Credits

This library was written primarily by Daniel Dugas. The transformer block codes, and vae/lstm code were taken or heavily derived from world models and karpathy's mingpt. We've retained the copyright headers for the relevant files.

navrep's Issues

test in ianenv

Hi, thanks for the code. I think your work is interesting and I am following your papers (soadrl, navrep and navdreams). There seems to be no checkpoint file for soadrl (where can I download this: soadrl/Final_models/angular_map_full_FOV). Could you please offer it to me for testing?

Thank you and wait for your reply~

AttributeError: module 'types' has no attribute 'CellType'

Hi,
I want to test out this environment with the pretrained models in your repository.
After I run "python -m navrep.scripts.cross_test_navreptrain_in_ianenv --backend VAE_LSTM --encoding V_ONLY --render", I get this:

2022-11-11 16:58:51.825066: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2022-11-11 16:58:51.835343: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2112000000 Hz 2022-11-11 16:58:51.837113: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x83e3410 executing computations on platform Host. Devices: 2022-11-11 16:58:51.837227: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined> 2022-11-11 16:58:51.931099: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile. 0%| | 0/3 [00:02<?, ?it/s] Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/tutu/navrepvenv/lib/python3.6/site-packages/navrep/scripts/cross_test_navreptrain_in_ianenv.py", line 101, in <module> c_model = PPO2.load(c_model_path) File "/home/tutu/navrepvenv/lib/python3.6/site-packages/stable_baselines/common/base_class.py", line 936, in load data, params = cls._load_from_file(load_path, custom_objects=custom_objects) File "/home/tutu/navrepvenv/lib/python3.6/site-packages/stable_baselines/common/base_class.py", line 666, in _load_from_file data = json_to_data(json_data, custom_objects=custom_objects) File "/home/tutu/navrepvenv/lib/python3.6/site-packages/stable_baselines/common/save_util.py", line 120, in json_to_data base64.b64decode(serialization.encode()) File "/home/tutu/navrepvenv/lib/python3.6/site-packages/cloudpickle/cloudpickle_fast.py", line 384, in <module> class CloudPickler(Pickler): File "/home/tutu/navrepvenv/lib/python3.6/site-packages/cloudpickle/cloudpickle_fast.py", line 406, in CloudPickler dispatch[types.CellType] = _cell_reduce **AttributeError: module 'types' has no attribute 'CellType'**

Is there an error in the version of my installation? I hope to get your help. Thank you~~

Issues when trying out test environment with navrep

Hi, so I've tried installing navrep and testing out this environment with the pretrained models in your repository- I have a couple issues that I was wondering about?
I've tried first updating any dependencies and I don't have permission to clone anything (e.g. deep_social_planner, etc).

Also, I run intro a strange issue where "dynamic module does not define module export function (PyInit__tf2) when trying out the test environment with navrep. It appears to be an issue due to the installed ros version being only compatible with python2.7?

Please let me know of your thoughts. Thanks

Bug in the simulation

There is a bug that happens almost always in any randomly generated map which is that the Lidar sensor detect that there is a wall but the robot can pass through the wall easily.
In other word the hit-box of the Lidar arrays is different from the the hit-box of the wall which leads to defected data and results.
The problem exist in the original code of the repository and no one noticed it even though it exist in almost every map generated (maybe because everyone running it in Reinforcement Learning models in headless mode so no one ever inspected anything manually).
I discovered this problem because i made an algorithm to navigate through the environment using path planning and re-planning using A-Star and some times the algorithm find a path through the wall and the robot passes through the wall without colliding (meaning the episode does not reset).
I usually saves the episode as a video to inspect the behavior of the algorithm (to improve it) but many times i encountered this issue and then discovered that this bug exist in almost all episodes.
And here is a link to some episodes where the bug happened.
The link also include some useful statistics if someone want to replicate the episodes.
https://drive.google.com/drive/folders/1osTR1u2T_FcAcHtRVsES1Yb1jOXeTIcG?usp=sharing
These are just an example there are many more.
In order to debug the problem i rendered the collision hit-box and this is what i found.
The dark area is the collision hit-box and the lines is the Lidar hit-box.

As you can see almost all the collision hit-boxes does not match the Lidar hit-box however some more than others (especially the ones that are close to the edges of the map)
After a lot of debugging i found out that when creating the obstruction it assigns the collision hit-box and Lidar hit-box.
Then when merging them with the map it checks if a part of the obstruction is outside the map so that it cuts the part that is outside the map from the obstruction without updating the new vertices of the Lidar hit-box which leads to the bug.
The solution to is bug is the following:
In navrepvenv\lib\python3.6\site-packages\crowd_sim\envs\crowed_sim.py
In the method generate_static_map_input()
In the last part of the method replace the following for loop:

        for obstacle_num, obstacle in enumerate(obstacles):
            if obstacle.location_x > obstacle.dim[0] / 2.0 and \
                obstacle.location_x < grid_size - obstacle.dim[0] / 2.0 and \
                obstacle.location_y > obstacle.dim[1] / 2.0 and \
                obstacle.location_y < grid_size - obstacle.dim[1] / 2.0:
                
                start_idx_x = int(
                    round(
                        obstacle.location_x -
                        obstacle.dim[0] /
                        2.0))
                start_idx_y = int(
                    round(
                        obstacle.location_y -
                        obstacle.dim[1] /
                        2.0))
                self.map[start_idx_x:start_idx_x +
                         obstacle.dim[0], start_idx_y:start_idx_y +
                         obstacle.dim[1]] = np.minimum(self.map[start_idx_x:start_idx_x +
                                                                obstacle.dim[0], start_idx_y:start_idx_y +
                                                                obstacle.dim[1]], obstacle.patch)
            else:
                x_test = []
                y_test = []
                for idx_x in range(obstacle.dim[0]):
                    for idx_y in range(obstacle.dim[1]):
                        shifted_idx_x = idx_x - obstacle.dim[0] / 2.0
                        shifted_idx_y = idx_y - obstacle.dim[1] / 2.0
                        submap_x = int(
                            round(
                                obstacle.location_x +
                                shifted_idx_x))
                        submap_y = int(
                            round(
                                obstacle.location_y +
                                shifted_idx_y))
                        if submap_x > 0 and submap_x < grid_size and submap_y > 0 and submap_y < grid_size:
                            #self.test_obs.append([submap_x, submap_y])
                            x_test.append(submap_x)
                            y_test.append(submap_y)
                            self.map[submap_x,
                                     submap_y] = obstacle.patch[idx_x, idx_y]

with the following updated one (where we update the vertices of the Lidar hit-box)

        self.test_obs = []                                                       
        for obstacle_num, obstacle in enumerate(obstacles):
            if obstacle.location_x > obstacle.dim[0] / 2.0 and \
                obstacle.location_x < grid_size - obstacle.dim[0] / 2.0 and \
                obstacle.location_y > obstacle.dim[1] / 2.0 and \
                obstacle.location_y < grid_size - obstacle.dim[1] / 2.0:
                
                start_idx_x = int(
                    round(
                        obstacle.location_x -
                        obstacle.dim[0] /
                        2.0))
                start_idx_y = int(
                    round(
                        obstacle.location_y -
                        obstacle.dim[1] /
                        2.0))
                self.map[start_idx_x:start_idx_x +
                         obstacle.dim[0], start_idx_y:start_idx_y +
                         obstacle.dim[1]] = np.minimum(self.map[start_idx_x:start_idx_x +
                                                                obstacle.dim[0], start_idx_y:start_idx_y +
                                                                obstacle.dim[1]], obstacle.patch)
                # Here we save the vertices
                self.test_obs.append([range(start_idx_x, start_idx_x + obstacle.dim[0]), range(start_idx_y, start_idx_y + obstacle.dim[1])])

            else:
                x_test = []
                y_test = []
                for idx_x in range(obstacle.dim[0]):
                    for idx_y in range(obstacle.dim[1]):
                        shifted_idx_x = idx_x - obstacle.dim[0] / 2.0
                        shifted_idx_y = idx_y - obstacle.dim[1] / 2.0
                        submap_x = int(
                            round(
                                obstacle.location_x +
                                shifted_idx_x))
                        submap_y = int(
                            round(
                                obstacle.location_y +
                                shifted_idx_y))
                        if submap_x > 0 and submap_x < grid_size and submap_y > 0 and submap_y < grid_size:
                            #self.test_obs.append([submap_x, submap_y])
                            x_test.append(submap_x)
                            y_test.append(submap_y)
                            self.map[submap_x,
                                     submap_y] = obstacle.patch[idx_x, idx_y]
                # Here we save the vertices
                self.test_obs.append([x_test, y_test])
            # Here where we update the vertices 
            obs = self.test_obs[obstacle_num].copy()
            self.obstacle_vertices[obstacle_num] = [((obs[0][-1] - grid_size/2)/10, (obs[1][0] - grid_size/2)/10),
                                                    ((obs[0][0] - grid_size/2)/10, (obs[1][0] - grid_size/2)/10),
                                                    ((obs[0][0] - grid_size/2)/10, (obs[1][-1] - grid_size/2)/10),
                                                    ((obs[0][-1] - grid_size/2)/10, (obs[1][-1] - grid_size/2)/10)]

And This is the result for the same episode shown above after fixing the problem. As we can see the vertices of the Lidar hit-box is updated to match the collision hit-box.

(Additional Info)
If you want to render the hit-boxes as in the images above do the following.
After fixing the problem we have an attribute of the soadrl_sim self.test_obs
In navrepvenv\lib\python3.6\site-packages\navrep\envs\navreptrainenv.py
In the render() method add this (somewhere in the middle)

def render()
    ...
 # hitbox
                for obs in self.soadrl_sim.test_obs:
                    gl.glBegin(gl.GL_QUADS)
                    gl.glColor4f(0, 0, 0, 0.3)
                    gl.glVertex3f((obs[0][-1] - self.grid_size/2)/10, (obs[1][0] - self.grid_size/2)/10, 0)
                    gl.glVertex3f((obs[0][0] - self.grid_size/2)/10, (obs[1][0] - self.grid_size/2)/10, 0)
                    gl.glVertex3f((obs[0][0] - self.grid_size/2)/10, (obs[1][-1] - self.grid_size/2)/10, 0)
                    gl.glVertex3f((obs[0][-1] - self.grid_size/2)/10, (obs[1][-1] - self.grid_size/2)/10, 0)
                    gl.glEnd()
    ...

If you want to save a video of the simulation you need to do the following:

In navrepvenv\lib\python3.6\site-packages\navrep\envs\navreptrainenv.py
In the reset() method add this at the end (just before the return)

def reset():
    ...
    self.video = []
    return ...

And in the the last part of the render method add this

def render:
    ...
frame = np.asarray(pyglet.image.get_buffer_manager().get_color_buffer().get_image_data().get_data()).reshape((WINDOW_W, WINDOW_H, 4))[::-1]
                #Image.fromarray(frame, 'RGBA').save("finally.png")
                self.video.append(frame)
   return ...

Now that we have the environment attribute self.video contains all the frames but it needs to be exported.
Here is the function that exports it to mp4 format.
Just put it somewhere and give it the environment attribute self.video and environment fps (so that the video will be real time fps=1/self._get_dt()) and the directory where the video will be saved.

def make_video(frames_array, fps, save_file):
        import cv2
        print(f"saving video to {save_file} ...")
        hieght = frames_array[0].shape[0]
        width = frames_array[0].shape[1]
        channel = frames_array[0].shape[2]
        fourcc = cv2.VideoWriter_fourcc(*'mp4v')
        video = cv2.VideoWriter(save_file, fourcc, float(fps), (width, hieght))
        for frame in frames_array:
            video.write(frame[:, :, 2::-1])
        video.release()
        print("done saving video")

And I hope this was helpful.
Best Regards