sigmabm/starcraft

Implementations of QMIX, VDN, COMA, QTRAN, CommNet, DyMA-CL, G2ANet on SMAC, the decentralised micromanagement scenario of StarCraft II

StarCraft

This is a pytorch implementation of the multi-agent reinforcement learning algorithms, including QMIX, VDN, COMA, QTRAN(both QTRAN-base and QTRAN-alt), and DyMA-CL, which are the state of art MARL algorithms. In addition, we implemented CommNet and combined it with coma, which we called CommNet_COMA. We trained these algorithms on SMAC, the decentralised micromanagement scenario of StarCraft II.

Corresponding Papers

• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Value-Decomposition Networks For Cooperative Multi-Agent Learning
• Counterfactual Multi-Agent Policy Gradients
• QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
• Learning Multiagent Communication with Backpropagation
• From Few to More: Large-scale Dynamic Multiagent Curriculum Learning

Requirements

• python=3.6.5
• torch=1.2.0
• SMAC
• pysc2

Acknowledgement

• SMAC
• pymarl

Quick Start

$ python main.py --evaluate_epoch=100 --map=3m --alg=qmix

Directly run the main.py, then the algorithm will be tested on map '3m' for 100 episodes, using the pretrained model.

The running of DyMA-CL is independent from others beacuse it requires different environment settings, you should open it as a new project, for more details, please read DyMA-CL documentation.

Result

We independently train these algorithms for 8 times and take the mean of the 8 independent results. In order to make the curves smoother, we also take the mean of every five points in the horizontal direction. In each independent training process, we train these algorithms for 5000 epochs and evaluate them for every 5 epochs. From the figure 1 we can see that our results is not the same as in the papers, maybe there are some small bugs, we are pleasure that you pull request to improve this project. Furthermore, as show in figure 2, we compare the best result we think in the 8 independent results. All of the results are saved in ./result .

1. Mean Win Rate of 8 Independent Runs on '3m'

2. Best Win Rate of 8 Independent Runs on '3m'