Raw Source and Filter Modelling for Dysarthric Speech Recognition

Yue, Zhengjun; Loweimi, Erfan; Cvetkovic, Zoran

Abstract

Acoustic modelling for automatic dysarthric speech recognition (ADSR) is a challenging task. Data deficiency is a major problem and substantial differences between the typical and dysarthric speech complicates transfer learning. In this paper, we build acoustic models using the raw magnitude spectra of the source and filter components. The proposed multi-stream model consists of convolutional and recurrent layers. It allows for fusing the vocal tract and excitation components at different levels of abstraction and after per-stream pre-processing. We show that such a multi-stream processing leverages these two information streams and helps s model towards normalising the speaker attributes and speaking style. This potentially leads to better handling of the dysarthric speech with a large inter-speaker and intra-speaker variability. We compare the proposed system with various features, study the training dynamics, explore usefulness of the data augmentation and provide interpretation for the learned convolutional filters. On the widely used TORGO dysarthric speech corpus, the proposed approach results in up to 1.7% absolute WER reduction for dysarthric speech compared with the MFCC base-line. Our best model reaches up to 40.6% and 11.8% WER for dysarthric and typical speech, respectively.

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• Start Date: May 23, 2022
• End Date: May 27, 2022
• Online Publication Date: Apr 27, 2022
• Publication Date: 2022
• Deposit Date: Apr 3, 2024
• Publisher: Institute of Electrical and Electronics Engineers
• Series ISSN: 2379-190X
• Book Title: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• DOI: https://doi.org/10.1109/icassp43922.2022.9746553
• Keywords: Dysarthric speech recognition, source-filter separation and fusion, multi-stream acoustic modelling
• Public URL: http://researchrepository.napier.ac.uk/Output/3585825