zhangzhao156/decentralized_qlearning_resource_allocation_in_wns

Implications of Decentralized Q-learning Resource Allocation in Wireless Networks

Authors

• Francesc Wilhelmi
• Boris Bellalta
• Cristina Cano
• Anders Jonsson

Project description

Reinforcement Learning is gaining attention by the wireless networking community due to its potential to learn good-performing configurations only from the observed results. In this work we propose a stateless variation of Q-learning, which we apply to exploit spatial reuse in a wireless network. In particular, we allow networks to modify both their transmission power and the channel used solely based on the experienced throughput. We concentrate in a completely decentralized scenario in which no information about neighbouring nodes is available to the learners. Our results show that although the algorithm is able to find the best-performing actions to enhance aggregate throughput, there is high variability in the throughput experienced by the individual networks. We identify the cause of this variability as the adversarial setting of our setup, in which the most played actions provide intermittent good/poor performance depending on the neighbouring decisions. We also evaluate the effect of the intrinsic learning parameters of the algorithm on this variability.

Repository description

This repository contains the Code and the LaTeX files used for the journal article "Implications of Decentralized Q-learning Resource Allocation in Wireless Networks", which was presented on the IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, Montréal 2017.

Running instructions

To run the code, just

1. "add to path" all the folders and subfolders in "./Code",
2. Access to "Code" folder
3. Execute scripts in folder "./Code/Experiments"

Contribute

If you want to contribute, please contact to [email protected]