MhaGNN: A novel framework for wearable sensor-based human activity recognition combining multi-head attention and graph neural networks

Wang, Yan; Wang, Xin; Yang, Hongmei; Geng, Yingrui; Yu, Hongnian; Zheng, Ge; Liao, Liang

Abstract

Obtaining robust feature representations from multi-position wearable sensory data is challenging in human activity recognition (HAR) since data from different positions can have unordered implicit correlations. Graph neural networks (GNNs) represent data as structured graphs by mining complex relationships and interdependency via message passing between the nodes of graphs. This paper proposes a novel framework (MhaGNN) that combines GNNs and the multi-head attention mechanism, aiming to learn more informative representations for multi-position HAR tasks. The MhaGNN framework takes the sensor channels from multiple wearing positions as nodes to construct graph-structured data from the spatial dimension. Besides, the multi-head attention mechanism is introduced to complete the message passing and aggregation of the graphs for spatial-temporal feature extraction. The MhaGNN learns correlations among sensor channels that can be used as compensatory features together with the captured features from each single sensor channel to enhance HAR. Experimental evaluations on three publicly available HAR datasets and a ground-truth dataset demonstrate that our proposed MhaGNN achieves state-of-the-art recognition performance with the captured rich features, including PAMAP2, OPPORTUNITY, MHAEATH and MPWHAR.

Citation

Wang, Y., Wang, X., Yang, H., Geng, Y., Yu, H., Zheng, G., & Liao, L. (2023). MhaGNN: A novel framework for wearable sensor-based human activity recognition combining multi-head attention and graph neural networks. IEEE Transactions on Instrumentation and Measurement, 72, Article 2514314. https://doi.org/10.1109/tim.2023.3276004

• Journal Article Type: Article
• Acceptance Date: Apr 28, 2023
• Online Publication Date: May 15, 2023
• Publication Date: 2023
• Deposit Date: Jun 28, 2023
• Journal: IEEE Transactions on Instrumentation and Measurement
• Print ISSN: 0018-9456
• Electronic ISSN: 1557-9662
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 72
• Article Number: 2514314
• DOI: https://doi.org/10.1109/tim.2023.3276004
• Keywords: Human activity recognition, graph neural network, attention mechanism, feature extraction, multi-position wearable sensors