Kaidi Xu
Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet
Xu, Kaidi; Huynh, Nguyen Van; Li, Geoffrey Ye
Authors
Nguyen Van Huynh
Geoffrey Ye Li
Abstract
In heterogeneous networks (HetNets), the overlap of small cells and the macro cell causes severe cross-tier interference. Although there exist some approaches to address this problem, they usually require global channel state information, which is hard to obtain in practice, and get the sub-optimal power allocation policy with high computational complexity. To overcome these limitations, we propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet, where each access point makes power control decisions independently based on local information. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. By introducing regularization terms in the loss function, each agent tends to choose an experienced action with high reward when revisiting a state, and thus the policy updating speed slows down. In this way, an agent’s policy can be learned by other agents more easily, resulting in a more efficient collaboration process. We then implement the proposed PQL in the considered HetNet and compare it with other distributed-training-and-execution (DTE) algorithms. Simulation results show that our proposed PQL can learn the desired power control policy from a dynamic environment where the locations of users change episodically and outperform existing DTE MADRL algorithms.
Citation
Xu, K., Huynh, N. V., & Li, G. Y. (2023). Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet. IEEE Transactions on Communications, 71(10), 5893 - 5903. https://doi.org/10.1109/tcomm.2023.3300331
Journal Article Type | Article |
---|---|
Acceptance Date | Jul 22, 2023 |
Online Publication Date | Aug 1, 2023 |
Publication Date | 2023-10 |
Deposit Date | Aug 5, 2023 |
Publicly Available Date | Aug 7, 2023 |
Journal | IEEE Transactions on Communications |
Print ISSN | 0090-6778 |
Electronic ISSN | 1558-0857 |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 71 |
Issue | 10 |
Pages | 5893 - 5903 |
DOI | https://doi.org/10.1109/tcomm.2023.3300331 |
Keywords | HetNet, distributed power control, multi-agent reinforcement learning, cooperative games |
Public URL | http://researchrepository.napier.ac.uk/Output/3160559 |
Files
Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet (accepted version)
(609 Kb)
PDF
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search