Abhijeet Bishnu
A Novel Frame Structure for Cloud-Based Audio-Visual Speech Enhancement in Multimodal Hearing-aids
Bishnu, Abhijeet; Gupta, Ankit; Gogate, Mandar; Dashtipour, Kia; Adeel, Ahsan; Hussain, Amir; Sellathurai, Mathini; Ratnarajah, Tharmalingam
Authors
Ankit Gupta
Dr. Mandar Gogate M.Gogate@napier.ac.uk
Senior Research Fellow
Dr Kia Dashtipour K.Dashtipour@napier.ac.uk
Lecturer
Ahsan Adeel
Prof Amir Hussain A.Hussain@napier.ac.uk
Professor
Mathini Sellathurai
Tharmalingam Ratnarajah
Abstract
In this paper, we design a first of its kind transceiver (PHY layer) prototype for cloud-based audio-visual (AV) speech enhancement (SE) complying with high data rate and low latency requirements of future multimodal hearing assistive technology. The innovative design needs to meet multiple challenging constraints including up/down link communications, delay of transmission and signal processing, and real-time AV SE models processing. The transceiver includes device detection, frame detection, frequency offset estimation, and channel estimation capabilities. We develop both uplink (hearing aid to the cloud) and downlink (cloud to hearing aid) frame structures based on the data rate and latency requirements. Due to the varying nature of uplink information (audio and lip-reading), the uplink channel supports multiple data rate frame structure, while the downlink channel has a fixed data rate frame structure. In addition, we evaluate the latency of different PHY layer blocks of the transceiver for developed frame structures using LabVIEW NXG. This can be used with software defined radio (such as Universal Software Radio Peripheral) for real-time demonstration scenarios.
Presentation Conference Type | Conference Paper (Published) |
---|---|
Conference Name | 2022 IEEE International Conference on E-health Networking, Application & Services (HealthCom) |
Start Date | Oct 17, 2022 |
End Date | Oct 19, 2022 |
Online Publication Date | Dec 21, 2022 |
Publication Date | 2022 |
Deposit Date | Apr 19, 2024 |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Pages | 75-80 |
Book Title | 2022 IEEE International Conference on E-health Networking, Application & Services (HealthCom) |
DOI | https://doi.org/10.1109/healthcom54947.2022.9982772 |
Public URL | http://researchrepository.napier.ac.uk/Output/3597124 |
You might also like
Toward's Arabic multi-modal sentiment analysis
(2018)
Presentation / Conference Contribution
A novel brain-inspired compression-based optimised multimodal fusion for emotion recognition
(2018)
Presentation / Conference Contribution
DNN driven speaker independent audio-visual mask estimation for speech separation
(2018)
Presentation / Conference Contribution
Deep learning driven multimodal fusion for automated deception detection
(2018)
Presentation / Conference Contribution
Exploiting Deep Learning for Persian Sentiment Analysis
(2018)
Presentation / Conference Contribution
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search