xushenlz/conflict_rez

Conflict resolution for multiple vehicles in tight spaces

conflict_rez

Conflict resolution for multiple vehicles in confined spaces

Xu Shen ([email protected]), Francesco Borrelli

Project Webpage: https://bit.ly/rl-cr

Install

1. Clone this repo
2. Run pip install -e . in the root level of this repo (A virtualenv is recommended). However, due to some buggy dependencies between package versions, you need to manually switch the following packages to specific versions (while ignoring the warning given by the pip dependency resolver):
   1. pip install pettingzoo==1.20.1
   2. pip install supersuit==3.5.0
   3. pip install stable-baselines3==1.6.0
   4. pip install gym==0.25.0

Testing

1. A pretrained model can be downloaded here.
2. Run python confrez/rl/experiment.py to see the steps taken by the DQN policy to resolve the conflict in the discrete environment.
3. Run python confrez/rl/record_states_history.py to generate a .pkl file that records the steps taken by the RL agents, which will serve as the configuration strategies for the trajectory planning problems.
4. Run python confrez/control/vehicle.py to plan a single-vehicle collision free trajectory following the strategy-guided configurations.
5. (Centralized method) Run python confrez/control/multi_vehicle_planner.py to solve the multi-vehicle trajectory planning problem to resolve conflict.
6. (Distributed method) Run python confrez/control/vehicle_follower.py so that each vehicle generates its own strategy-guided reference trajectory, and then follows it with distributed MPC to avoid collisions.

Training

1. Run python confrez/rl/train.py to train a new policy.
2. You may set the random_reset argument to True so that you train the policy with a random subset out of 4 vehicles, which may lead to a more generalizable policy. But it also require longer training time.
3. Different random seeds and different parameter tuning will lead to different behaviors in the trained policy. With your new policy, the vehicles may (almost for sure) take different actions and resolve the conflict in another way.

conflict_rez's Issues

Feature request: update PettingZoo version

Hi, would it be possible to update this repo to use the most recent version of PettingZoo? We want to list this project in PettingZoo's third-party-environments, but we can only include environments which work with the current version.

If you need any help working out issues due to different versions feel free to ask, there were some breaking changes in version 1.2, so it requires a bit of code changes to adapt. The previous API returned done in the step() function, whereas the new one returns truncated and terminated (matching gymnasium). There is a migration guide for gymnasium explaining the changes further, the steps should be basically the same (we're working on making resources for updating old PettingZoo repositories as well): https://gymnasium.farama.org/content/migration-guide/