mit-realm/macbf

Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates

Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates

Zengyi Qin, Kaiqing Zhang, Yuxiao Chen, Jingkai Chen, Chuchu Fan

This repository contains the official implementation of Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates published at the International Conference on Learning Representations (ICLR), 2021.

Installation

Create a virtual environment with Anaconda:

conda create -n macbf python=3.6

Activate the virtual environment:

source activate macbf

Clone this repository:

git clone https://github.com/Zengyi-Qin/macbf.git

Enter the main folder and install the dependencies:

pip install -r requirements.txt

Cars

In cars, we provide a multi-agent collision avoidance example with the double integrator dynamics. First enter the directory:

cd cars

To evaluate the pretrained neural network CBF and controller, run:

python evaluate.py --num_agents 32 --model_path models/model_save --vis 1

--num_agents specifies the number of agents in the environment. --model_path points to the prefix of the pretrained neural network weights. The visualization is disabled by default and will be enabled when --vis is set to 1.

To train the neural network CBF and controller from scratch, run:

python train.py --num_agents 32

We can add another argument --model_path and point to a pretrained model we want to use.

Drones

In drones, we consider the drone dynamics with 8-dimsional state space. Details of the dynamics can be found in Appendix C of our paper. To experiment with this example, first enter the directory:

cd drones

To evaluate the pretrained neural network CBF and controller, run:

python evaluate.py --num_agents 32 --model_path models/model_save --vis 1

To train the neural network CBF and controller from scratch, run:

python train.py --num_agents 32

The arguments are the same as the cars example.

macbf's Issues

What consists of the agent observation in MACBF?

I'm curious about how to construct the scenarios used in your ICLR paper. Meanwhile, is the collision with the obstacles like the wall of the Maze not regarded as an unsafe state? Can the agents observe the states of different scenarios, for example, the shape, size and location of the wall?

Neural Barrier Certificates for reinforcement learning tasks with discrete actions

Hello @Zengyi-Qin @jkchengh @kzhang66

I have read your paper Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates. The idea to combine the loss function for control barrier functions and the controller is fascinating.

As far as I understood, you have experimented with a task having continuous action space. Can this be applied to a reinforcement learning task with discrete action space?