Novel Two-Stage Audiovisual Speech Filtering in Noisy Environments

Abel, Andrew; Hussain, Amir

Abstract

In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, we build on previous work by the authors and present a novel two-stage audiovisual speech enhancement system, making use of audio-only beamforming, automatic lip tracking, and pre-processing with visually derived Wiener speech filtering. Initial results have demonstrated that this two-stage multimodal speech enhancement approach can produce positive results with noisy speech mixtures that conventional audio-only beamforming would struggle to cope with, such as in very noisy environments with a very low signal to noise ratio, and when the type of noise is difficult for audio-only beamforming to process.

Citation

Abel, A., & Hussain, A. (2014). Novel Two-Stage Audiovisual Speech Filtering in Noisy Environments. Cognitive Computation, 6(2), 200-217. https://doi.org/10.1007/s12559-013-9231-2

• Journal Article Type: Article
• Acceptance Date: Oct 1, 2013
• Online Publication Date: Oct 20, 2013
• Publication Date: 2014
• Deposit Date: Sep 26, 2019
• Journal: Cognitive Computation
• Print ISSN: 1866-9956
• Electronic ISSN: 1866-9964
• Publisher: BMC
• Peer Reviewed: Peer Reviewed
• Volume: 6
• Issue: 2
• Pages: 200-217
• DOI: https://doi.org/10.1007/s12559-013-9231-2
• Keywords: Speech enhancement; Multimodal speech filtering; Audiovisual speech processing
• Public URL: http://researchrepository.napier.ac.uk/Output/1793059