Q-learning driven routing for aeronautical Ad-Hoc networks

Bilen, Tuğçe; Canberk, Berk

Abstract

The aeronautical ad-hoc network (AANET) is one of the critical methodologies to satisfy the Internet connectivity requirement of airplanes during their flights. However, the ultra-dynamic topology and unstable air-to-air link characteristics increase the need for AANETs to a particular routing algorithm compared to the terrestrial networks. This need is mainly because these AANET-specific characteristics increase the delays, packet losses, and network load with accuracy reduction by continuously changing topology and breaking air-to-air links during routing. The works in the literature do not satisfy the ultra-dynamic topology and unstable air-to-air link characteristics of AANETs during routing. On the other hand, the routing algorithm can adapt to the dynamic conditions of AANETs by utilizing Artificial Intelligence (AI) based methodologies. For adaptation to this dynamic environment, we aim to let the airplanes find their routing path through exploration and exploitation by mapping the AANET environment to QLR. Clearly, this article proposes an updated Layered Hidden Markov Model (updated-LHMM) estimation-based Q-learning (QLR) routing for AANETs to solve the delay, packet loss, network load, and accuracy problems. For this aim, the Bellman Equation is adapted to the AANET environment by proposing different methodologies for its related QLR components. Results reveal that the proposed strategy mainly reduces the routing delay and packet losses by 30% and 33% compared to the methods in the literature.

Citation

Bilen, T., & Canberk, B. (2022). Q-learning driven routing for aeronautical Ad-Hoc networks. Pervasive and Mobile Computing, 87, Article 101724. https://doi.org/10.1016/j.pmcj.2022.101724

• Journal Article Type: Article
• Acceptance Date: Nov 7, 2022
• Online Publication Date: Nov 11, 2022
• Publication Date: 2022-12
• Deposit Date: Nov 15, 2022
• Publicly Available Date: Nov 12, 2023
• Journal: Pervasive and Mobile Computing
• Print ISSN: 1574-1192
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 87
• Article Number: 101724
• DOI: https://doi.org/10.1016/j.pmcj.2022.101724
• Keywords: AANETs, Routing management, Reinforcement learning, Q-learning, Hidden Markov model
• Public URL: http://researchrepository.napier.ac.uk/Output/2957850