Yanhong Liu
A Transformer-based Gesture Prediction Model via sEMG Sensor for Human-robot Interaction
Liu, Yanhong; Li, Xingyu; Yang, Lei; Yu, Hongnian
Abstract
As one of the most direct and pivotal modes of human-computer interaction (HCI), the application of surface electromyography (sEMG) signals in the domain of gesture prediction has emerged as a prominent area of research. To enhance the performance of gesture prediction system based on multi-channel sEMG signals, a novel gesture prediction framework is proposed that (i) Conversion of original biological signals from multi-channel sEMG into two-dimensional time-frequency maps is achieved through the incorporation of continuous wavelet transform (CWT). (ii) For two-dimensional time-frequency map inputs, a Transformer-based classification network that effectively learns local and global context information is proposed, named DIFT-Net, with the goal of implementing sEMG-based gesture prediction for robot interaction. Proposed DIFT-Net employs a dual-branch interactive fusion structure based on the Swin Transformer, enabling effective acquisition of global contextual information and local details. Additionally, an attention guidance module (AGM) and an attentional interaction module (AIM) are proposed to guide network feature extraction and fusion processes in proposed DIFT-Net. The AGM module takes intermediate features from the same stage of both branches as input and guides the network to extract more localized and detailed features through convolutional attention. Meanwhile, the AIM module integrates output features from both branches to enhance the aggregation of global context information across various scales. To substantiate the efficacy of DIFT-Net, a multi-channel EMG bracelet is utilized to collect and construct an sEMG signal dataset. Experimental results demonstrate that the proposed DIFT-Net attains an accuracy of 98.36% in self-built dataset and 82.64% accuracy on the public Nanapro DB1 dataset.
Citation
Liu, Y., Li, X., Yang, L., & Yu, H. (2024). A Transformer-based Gesture Prediction Model via sEMG Sensor for Human-robot Interaction. IEEE Transactions on Instrumentation and Measurement, 73, Article 2510615. https://doi.org/10.1109/tim.2024.3373045
Journal Article Type | Article |
---|---|
Acceptance Date | Feb 15, 2024 |
Online Publication Date | Mar 4, 2024 |
Publication Date | 2024-03 |
Deposit Date | Mar 11, 2024 |
Journal | IEEE Transactions on Instrumentation and Measurement |
Print ISSN | 0018-9456 |
Electronic ISSN | 1557-9662 |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 73 |
Article Number | 2510615 |
DOI | https://doi.org/10.1109/tim.2024.3373045 |
Keywords | Hand Gesture Recognition, sEMG Sensor, Transformer, Feature Fusion, Human-robot Interaction |
Public URL | http://researchrepository.napier.ac.uk/Output/3535881 |
You might also like
Predicting the relationships between virtual enterprises and agility in supply chains
(2017)
Journal Article
A practical multi-sensor activity recognition system for home-based care
(2014)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search