Umer Saeed
Extracting Visual Micro-Doppler Signatures from Human Lips Motion Using UoG Radar Sensing Data for Hearing Aid Applications
Saeed, Umer; Shah, Syed Aziz; Ghadi, Yazeed Yasin; Khan, Muhammad Zakir; Ahmad, Jawad; Shah, Syed Ikram; Hameed, Hira; Abbasi, Qammer H.
Authors
Syed Aziz Shah
Yazeed Yasin Ghadi
Muhammad Zakir Khan
Dr Jawad Ahmad J.Ahmad@napier.ac.uk
Associate
Syed Ikram Shah
Hira Hameed
Qammer H. Abbasi
Abstract
This study proposes a secure and effective lips-reading system that can accurately detect lips movements, even when face masks are worn. The system utilizes radio frequency (RF) sensing and ultra-wideband (UWB) radar technology, which overcomes the challenges posed by traditional vision-based systems. By leveraging deep learning models, the system interprets lips and mouth movements and achieves an overall accuracy of 90% for both mask-on and mask-off scenarios. The study utilized a trusted dataset from the University of Glasgow (UoG), consisting of spectrograms of lips motions stating five vowels and a voiceless class from distinct participants. The cutting-edge deep learning algorithm, Residual Neural Network (ResNet50), was used for the evaluation of the dataset and achieved an 87% accurate detection rate with a mask-on scenario, which is a 14% improvement compared to prior published work. The findings of this study contribute to the development of a robust lips-reading framework that can enhance communication accessibility in applications such as hearing aids, voice-controlled systems, biometrics, and more.
Citation
Saeed, U., Shah, S. A., Ghadi, Y. Y., Khan, M. Z., Ahmad, J., Shah, S. I., Hameed, H., & Abbasi, Q. H. (2023). Extracting Visual Micro-Doppler Signatures from Human Lips Motion Using UoG Radar Sensing Data for Hearing Aid Applications. IEEE Sensors Journal, 23(19), 22111-22118. https://doi.org/10.1109/jsen.2023.3308972
Journal Article Type | Article |
---|---|
Acceptance Date | Aug 22, 2023 |
Online Publication Date | Aug 31, 2023 |
Publication Date | Oct 1, 2023 |
Deposit Date | Oct 17, 2023 |
Journal | IEEE Sensors Journal |
Print ISSN | 1530-437X |
Electronic ISSN | 1558-1748 |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 23 |
Issue | 19 |
Pages | 22111-22118 |
DOI | https://doi.org/10.1109/jsen.2023.3308972 |
Keywords | ResNet50; InceptionV3; VGG16; RF sensing; UWB radar; lips-reading; speech recognition |
You might also like
Transparent RFID tag wall enabled by artificial intelligence for assisted living
(2024)
Journal Article
Enhancing Cloud Computing Security Through Blockchain-Based Communication for Electronic Health Records
(2024)
Presentation / Conference Contribution
ML-Driven Attack Detection in RPL Networks: Exploring Attacker Position's Significance
(2024)
Presentation / Conference Contribution
Chaotic Quantum Encryption to Secure Image Data in Post Quantum Consumer Technology
(2024)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search