Lightweight Reinforcement Learning for Energy Efficient Communications in Wireless Sensor Networks

Savaglio, Claudio; Pace, Pasquale; Aloi, Gianluca; Liotta, Antonio; Fortino, Giancarlo

Abstract

High-density communications in wireless sensor networks (WSNs) demand for new approaches to meet stringent energy and spectrum requirements. We turn to reinforcement learning, a prominent method in artificial intelligence, to design an energy-preserving MAC protocol, with the aim to extend the network lifetime. Our QL-MAC protocol is derived from Q-learning, which iteratively tweaks the MAC parameters through a trial-and-error process to converge to a low energy state. This has a dual benefit of 1) solving this minimization problem without the need of predetermining the system model and 2) providing a self-adaptive protocol to topological and other external changes. QL-MAC self-adjusts the WSN node duty-cycle, reducing energy consumption without detrimental effects on the other network parameters. This is achieved by adjusting the radio sleeping and active periods based on traffic predictions and transmission state of neighboring nodes. Our findings are corroborated by an extensive set of experiments carried out on off-the-shelf devices, alongside large-scale simulations. INDEX TERMS Wireless sensor network, artificial intelligence, reinforcement learning, energy-efficient network, medium access control.

• Journal Article Type: Article
• Acceptance Date: Feb 20, 2019
• Online Publication Date: Mar 4, 2019
• Publication Date: 2019
• Deposit Date: Jul 29, 2019
• Journal: IEEE Access
• Electronic ISSN: 2169-3536
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 7
• Pages: 29355-29364
• DOI: https://doi.org/10.1109/access.2019.2902371
• Keywords: General Engineering; General Materials Science; General Computer Science
• Public URL: http://researchrepository.napier.ac.uk/Output/2006529