Optimal Beam Association in mmWave Vehicular Networks with Parallel Reinforcement Learning

Van Huynh, Nguyen; Nguyen, Diep N.; Hoang, Dinh Thai; Dutkiewicz, Eryk

Abstract

This paper develops a beam association framework for mm Wave vehicular networks to improve the system performance in terms of handover, disconnection time, and data rate under the high mobility of vehicles. In particular, we recruit the semi Markov decision process to capture the uncertainty and dynamic of the environment such as locations of beams, received signal strength indicator profiles, velocities, and blockages. Instead of adopting complex deep learning structures such as deep dueling and double deep Q-learning, we develop a lightweight yet very effective parallel Q-learning algorithm to quickly derive the optimal beam association policy by simultaneously learning from various vehicles on the road. Through extensive simulation results, we demonstrate that the proposed framework can reduce the average disconnection time by 33% and increase the data rate by 60% compared to other solutions. We also observed that the proposed parallel Q-learning algorithm converges much faster to the optimal solution than state-of-the-art deep-learning based algorithms.

Citation

Van Huynh, N., Nguyen, D. N., Hoang, D. T., & Dutkiewicz, E. (2020, December). Optimal Beam Association in mmWave Vehicular Networks with Parallel Reinforcement Learning. Presented at GLOBECOM 2020 - 2020 IEEE Global Communications Conference, Taipei, Taiwan

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: GLOBECOM 2020 - 2020 IEEE Global Communications Conference
• Start Date: Dec 7, 2020
• End Date: Dec 11, 2020
• Online Publication Date: Feb 15, 2021
• Publication Date: 2020
• Deposit Date: Mar 29, 2023
• Publisher: Institute of Electrical and Electronics Engineers
• Series ISSN: 2576-6813
• Book Title: GLOBECOM 2020 - 2020 IEEE Global Communications Conference
• DOI: https://doi.org/10.1109/globecom42002.2020.9348240