haiyang-w/mavn

Collaborative Visual Navigation

Introduction

CollaVN is a visual MARL dataset, created for the development of intelligent, cooperative navigation agents.

Authors: Haiyang Wang, Wenguan Wang, Jifeng Dai, Xizhou Zhu, Liwei Wang

[arxiv]

Latest Updates

[08/17/2021] Release CollaVN V1 dataset

Citation

If you use CollaVN and models, consider citing the following publication:

@article{wang2021collaborative,
  title={Collaborative Visual Navigation},
  author={Wang, Haiyang and Wang, Wenguan and Zhu, Xizhou and Dai, Jifeng and Wang, Liwei},
  journal={arXiv preprint arXiv:2107.01151},
  year={2021}
}

Installation

The code is tested under the following environment:

• Ubuntu 16.04 LTS
• Python 3.6
• Pytorch 1.8.0
• CUDA 11.1
• GCC 7.3

Install required package

conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge

pip install scikit-image

Install MAVN

git clone https://github.com/Haiyang-W/MAVN.git

cd MAVN

# if you didn't create the conda environment before:
conda create -y -n MAVN python=3.6
conda activate MAVN

pip install -e . 

Note

If you have problem in openGL, please see [Trouble Shooting]

Presample Datapoint

To make the comparison fair, we pre-generated some data points, which we used in our paper, please see dataset dir. Google Drive

cd MAVN
wget https://drive.google.com/drive/folders/1xYzl5zBYOfTE2ouSIKgp6ZgrQ7lZEe7X?usp=sharing

Run Demo

python demo.py -n1 --auto_gpu_config 1 --total_num_scenes 1 --task_config ./config/CommonGoal_two_locobot.yaml

TODO:

Release CollaVN V2 and method code

Acknowledgments

Our code is heavily based on iGibson. Thanks iGibson Development Team for their awesome codebase.

mavn's Issues

BrokenPipeError when trying to run the demo

Hi. I made tried running the demo following the instructions in the package's readme file, but I am faced with the following error:

(MAVN36) farjadnm@PC-SIROUSPOUR14:~/MAVN$ python demo.py -n1 --auto_gpu_config 1 --total_num_scenes 1 --task_config ./config/CommonGoal_two_locobot.yaml
INFO:root:Importing iGibson (gibson2 module)
INFO:root:Assets path: /home/farjadnm/MAVN/gibson2/./data/assets
INFO:root:Dataset path: /home/farjadnm/MAVN/gibson2/./data/g_dataset
pybullet build time: May 20 2022 19:41:54
/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/site-packages/gym/core.py:27: UserWarning: WARN: Gym minimally supports python 3.6 as the python foundation not longer supports the version, please update your version to 3.7+
"Gym minimally supports python 3.6 as the python foundation not longer supports the version, please update your version to 3.7+"
/home/farjadnm/.local/lib/python3.6/site-packages/torch/cuda/init.py:143: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Auto GPU config:
Number of processes: 1
Number of processes on GPU 0: 1
Number of processes per GPU: 1
INFO:root:Importing iGibson (gibson2 module)
INFO:root:Assets path: /home/farjadnm/MAVN/gibson2/./data/assets
INFO:root:Dataset path: /home/farjadnm/MAVN/gibson2/./data/g_dataset
pybullet build time: May 20 2022 19:41:54
/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/site-packages/gym/core.py:27: UserWarning: WARN: Gym minimally supports python 3.6 as the python foundation not longer supports the version, please update your version to 3.7+
"Gym minimally supports python 3.6 as the python foundation not longer supports the version, please update your version to 3.7+"
/home/farjadnm/.local/lib/python3.6/site-packages/torch/cuda/init.py:143: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Auto GPU config:
Number of processes: 1
Number of processes on GPU 0: 1
Number of processes per GPU: 1
INFO:root:Using device 0 for rendering
INFO:root:Using cuda device 0 for pytorch
PyBullet Logging Information:
PyBullet Logging Information
INFO:root:Building scene: Wainscott
Process ForkServerProcess-1:
Traceback (most recent call last):
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/contextlib.py", line 52, in inner
return func(*args, **kwds)
File "/home/farjadnm/MAVN/env/paralle_env.py", line 168, in _worker_env
env = env_fn(*env_fn_args) # single environment
File "/home/farjadnm/MAVN/env/paralle_env.py", line 57, in init_env
physics_timestep=1 / 40.0, render_to_tensor=render_to_tensor, device_idx=devices, rank=rank, seed=seed_num)
File "/home/farjadnm/MAVN/env/make_collavn_env.py", line 55, in init
rank=rank)
File "/home/farjadnm/MAVN/env/mabase_env.py", line 56, in init
render_to_tensor=render_to_tensor)
File "/home/farjadnm/MAVN/env/base_env.py", line 47, in init
self.load()
File "/home/farjadnm/MAVN/env/mabase_env.py", line 227, in load
super(MaBase_ImgGoal_NavigateEnv, self).load()
File "/home/farjadnm/MAVN/env/base_env.py", line 87, in load
self.simulator.import_scene(scene, load_texture=self.config.get('load_texture', True))
File "/home/farjadnm/MAVN/gibson2/core/simulator.py", line 135, in wrapped_load_func
res = load_func(*args, **kwargs)
File "/home/farjadnm/MAVN/gibson2/core/simulator.py", line 155, in import_scene
new_objects = scene.load()
File "/home/farjadnm/MAVN/gibson2/core/physics/scene.py", line 292, in load
self.load_floor_metadata()
File "/home/farjadnm/MAVN/gibson2/core/physics/scene.py", line 125, in load_floor_metadata
floor_height_path = os.path.join(get_model_path(self.model_id), 'floors.txt')
File "/home/farjadnm/MAVN/gibson2/utils/assets_utils.py", line 7, in get_model_path
assert model_id in os.listdir(data_path) or model_id == 'stadium', "Model {} does not exist".format(model_id)
FileNotFoundError: [Errno 2] No such file or directory: '/home/farjadnm/MAVN/gibson2/./data/g_dataset'
Traceback (most recent call last):
File "demo.py", line 78, in
main()
File "demo.py", line 57, in main
envs = make_parallel_env(args.task_config, args, data_split, render_to_tensor=render_to_tensor, mode='headless', gpu_id_list=gpu_id_list)
File "/home/farjadnm/MAVN/env/paralle_env.py", line 604, in make_parallel_env
envs = construct_envs(config_filename, args, scenes_split, mode, render_to_tensor, gpu_id_list)
File "/home/farjadnm/MAVN/env/paralle_env.py", line 595, in construct_envs
env_fn_args=args_list,
File "/home/farjadnm/MAVN/env/paralle_env.py", line 130, in init
read_fn() for read_fn in self._connection_read_fns
File "/home/farjadnm/MAVN/env/paralle_env.py", line 130, in
read_fn() for read_fn in self._connection_read_fns
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
Exception ignored in: <bound method ProcessPyEnvironment.del of <env.paralle_env.ProcessPyEnvironment object at 0x7f527c04ed30>>
Traceback (most recent call last):
File "/home/farjadnm/MAVN/env/paralle_env.py", line 479, in del
self.close()
File "/home/farjadnm/MAVN/env/paralle_env.py", line 369, in close
write_fn((CLOSE_COMMAND, None))
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/home/farjadnm/anaconda3/envs/MAVN36/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

I get the same error running this in an environment with python 3.7 as well.
Can you please help me fix this issue?
Thanks!

Training Time

Dear Authors,

How long does your model need to train?

Thanks