imanrht/qeco

A QoE-Oriented Computation Offloading Algorithm based on Deep Reinforcement Learning for Mobile Edge Computing

QECO

A QoE-Oriented Computation Offloading Algorithm based on Deep Reinforcement Learning for Mobile Edge Computing

This repository contains the Python code for reproducing the decentralized QOCO (QoE-Oriented Computation Offloading) algorithm, designed for Mobile Edge Computing systems. QOCO leverages Deep Reinforcement Learning to empower mobile devices to make their offloading decisions and select offloading targets, with the aim of maximizing the long-term Quality of Experience (QoE) for each user individually.

Contents

• main.py: The main code, including training and testing structures, implemented using Tensorflow 1.x.
• MEC_Env.py: Contains the code for the mobile edge computing environment.
• DDQN.py: The code for reinforcement learning with double deep Q networks for mobile devices, implemented using Tensorflow 1.x.
• DDQN_keras.py: Dueling double deep Q-network (D3QN) implementation using Keras.
• DDQN_torch.py: Dueling double deep Q-network (D3QN) implementation using PyTorch.
• Config.py: Configuration file for MEC entities and neural network setup.

Cite this Work

If you use this work in your research, please cite it as follows:

I. Rahmati, H. Shahmansouri, and A. Movaghar, "QECO: A QoE-Oriented Computation Offloading Algorithm based on Deep Reinforcement Learning for Mobile Edge Computing", submitted to IEEE Internet of Things Journal, Oct 2023.

@article{rahmati2023qeco,
  title={QECO: A QoE-Oriented Computation Offloading Algorithm based on Deep Reinforcement Learning for Mobile Edge Computing},
  author={Rahmati, Iman and Shah-Mansouri, Hamed and Movaghar, Ali},
  journal={arXiv preprint arXiv:2311.02525},
  year={2023}
}

About Authors

• Iman Rahmati: Research Assistant in the Computer Science and Engineering Department at SUT.
• Hamed Shah-Mansouri: Assistant Professor in the Electrical Engineering Department at SUT.
• Ali Movaghar: Professor in the Computer Science and Engineering Department at SUT.

Required Packages

Make sure you have the following packages installed:

• Tensorflow
• PyTorch
• numpy
• matplotlib

Primary References

• H. Shah-Mansouri and V. W. Wong, “Hierarchical fog-cloud computing for iot systems: A computation offloading game", IEEE Internet of Things Journal, May 2018.

• M. Tang and V. W. Wong, "Deep reinforcement learning for task offloading in mobile edge computing systems", IEEE Transactions on Mobile Computing, Nov 2020.

• H. Zhou, K. Jiang, X. Liu, X. Li, and V. C. Leung, “Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing”, IEEE Internet of Things Journal, Jun 2021.

• L. Yang, H. Zhang, X. Li, H. Ji, and V. C. Leung, “A distributed computation offloading strategy in small-cell networks integrated with mobile edge computing”, IEEE/ACM Transactions on Networking, Dec 2018.

Contribute

If you have an issue or found a bug, please raise a GitHub issue here. Pull requests are also welcome.

qeco's Issues

some questions about convergence

I've run your code, but the results didn't converge. I read your reward function, find that you only consider energy and drop rate? I've tried to modify the rewrd function, but the energy still didn't converge.

about the code of benchmark PGOA

Can you share your code of benchmark PGOA? Thanks!

MEC_Env.py

This code is good for me to learn MEC and DRL. And hope u can add MEC_Env.py to the code so that I can learn how to build environment of MEC. Thanks!

in DDQN_torch, line 146, q_next,q_eval4next = self.forward(...), it need two return values in forward() , but the forward() return only a q.

I think the value of the q_eval4next need a new line of code to get, could you please give a tip how to modify it ?

q_next, q_eval4next = self.forward(batch_memory[:, -self.n_features:], lstm_batch_memory[:, :, self.n_lstm_state:])

main.py

Hello, do you have a main file to run with pytorch? Could you share it? Thank you very much.

Running error

May I ask if the following error occurs when I import kears and torch respectively? How can I solve it? Please help me, which is very important to me

Help

I think the formula (10) in this article is wrong, could you help me? Thanks.