Reward-Reinforced Generative Adversarial Networks for Multi-agent Systems

Zheng, Changgang; Yang, Shufan; Parra-Ullauri, Juan; Garcia-Dominguez, Antonio; Bencomo, Nelly

Abstract

Multi-agent systems deliver highly resilient and adaptable solutions for common problems in telecommunications, aerospace, and industrial robotics. However, achieving an optimal global goal remains a persistent obstacle for collaborative multi-agent systems, where learning affects the behaviour of more than one agent. A number of nonlinear function approximation methods have been proposed for solving the Bellman equation, which describe a recursive format of an optimal policy. However, how to leverage the value distribution based on reinforcement learning, and how to improve the efficiency and efficacy of such systems remain a challenge. In this work, we developed a reward-reinforced generative adversarial network to represent the distribution of the value function, replacing the approximation of Bellman updates. We demonstrated our method is resilient and outperforms other conventional reinforcement learning methods. This method is also applied to a practical case study: maximising the number of user connections to autonomous airborne base stations in a mobile communication network. Our method maximises the data likelihood using a cost function under which agents have optimal learned behaviours. This reward-reinforced generative adversarial network can be used as a generic framework for multi-agent learning at the system level.

Citation

Zheng, C., Yang, S., Parra-Ullauri, J., Garcia-Dominguez, A., & Bencomo, N. (2022). Reward-Reinforced Generative Adversarial Networks for Multi-agent Systems. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(3), 479 - 488. https://doi.org/10.1109/TETCI.2021.3082204

• Journal Article Type: Article
• Acceptance Date: Apr 26, 2021
• Online Publication Date: Jun 8, 2021
• Publication Date: 2022-06
• Deposit Date: May 6, 2021
• Publicly Available Date: Jun 8, 2021
• Print ISSN: 2471-285X
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 6
• Issue: 3
• Pages: 479 - 488
• DOI: https://doi.org/10.1109/TETCI.2021.3082204
• Keywords: multi-agent; reinforcement learning; GAN; reward-reinforced GAN; airborne base station (ABS)
• Public URL: http://researchrepository.napier.ac.uk/Output/2756866