A Novel Speech Intelligibility Enhancement Model based on Canonical Correlation and Deep Learning

Hussain, Tassadaq; Diyan, Muhammad; Gogate, Mandar; Dashtipour, Kia; Adeel, Ahsan; Tsao, Yu; Hussain, Amir

Abstract

Current deep learning (DL) based approaches to speech intelligibility enhancement in noisy environments are often trained to minimise the feature distance between noise-free speech and enhanced speech signals. Despite improving the speech quality, such approaches do not deliver required levels of speech intelligibility in everyday noisy environments. Intelligibility-oriented (I-O) loss functions have recently been developed to train DL approaches for robust speech enhancement. Here, we formulate, for the first time, a novel canonical correlation based I-O loss function to more effectively train DL algorithms. Specifically, we present a canonical-correlation based short-time objective intelligibility (CC-STOI) cost function to train a fully convolutional neural network (FCN) model. We carry out comparative simulation experiments to show that our CC-STOI based speech enhancement framework outperforms state-of-the-art DL models trained with conventional distance-based and STOI-based loss functions, using objective and subjective evaluation measures for case of both unseen speakers and noises. Ongoing future work is evaluating the proposed approach for design of robust hearing-assistive technology.

Citation

Hussain, T., Diyan, M., Gogate, M., Dashtipour, K., Adeel, A., Tsao, Y., & Hussain, A. (2022, July). A Novel Speech Intelligibility Enhancement Model based on Canonical Correlation and Deep Learning. Presented at 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, Scotland

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)
• Start Date: Jul 11, 2022
• End Date: Jul 15, 2022
• Online Publication Date: Sep 8, 2022
• Publication Date: 2022
• Deposit Date: Apr 19, 2024
• Publisher: Institute of Electrical and Electronics Engineers
• Pages: 2581-2584
• Series ISSN: 2694-0604
• Book Title: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)
• DOI: https://doi.org/10.1109/embc48229.2022.9871113
• Public URL: http://researchrepository.napier.ac.uk/Output/3597136