Cognitively inspired speech processing for multimodal hearing technology

Abel, Andrew; Hussain, Amir; Luo, Bin

Abstract

In recent years, the link between the various human communication production domains has become more widely utilised in the field of speech processing. Work by the authors and others has demonstrated that intelligently integrated audio and visual information can be used for speech enhancement. This advance in technology means that the use of visual information as part of hearing aids or assistive listening devices is becoming ever more viable. One issue that is not commonly explored is how a multimodal system copes with variations in data quality and availability, such as a speaker covering their face while talking, or the existence of multiple speakers in a conversational scenario, an issue that a hearing device would be expected to cope with by switching between different programmes and settings to adapt to changes in the environment. We present the ChallengAV audiovisual corpus, which is used to evaluate a novel fuzzy logic based audiovisual switching system, designed to be used as part of a next-generation adaptive, autonomous, context aware hearing system. Initial results show that the detectors are capable of determining environmental conditions and responding appropriately, demonstrating the potential of such an adaptive multimodal system as part of a state of the art hearing aid device.

Citation

Abel, A., Hussain, A., & Luo, B. (2014, December). Cognitively inspired speech processing for multimodal hearing technology. Presented at 2014 IEEE Symposium on Computational Intelligence in Healthcare and e-health (CICARE), Orlando, FL, USA

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2014 IEEE Symposium on Computational Intelligence in Healthcare and e-health (CICARE)
• Start Date: Dec 9, 2014
• End Date: Dec 12, 2014
• Online Publication Date: Jan 15, 2015
• Publication Date: 2015
• Deposit Date: Oct 10, 2019
• Publisher: Institute of Electrical and Electronics Engineers
• Pages: 56-63
• Book Title: 2014 IEEE Symposium on Computational Intelligence in Healthcare and e-health (CICARE)
• DOI: https://doi.org/10.1109/CICARE.2014.7007834
• Keywords: Visualization, Speech, Input variables, Detectors, Fuzzy logic, Noise, Speech enhancement
• Public URL: http://researchrepository.napier.ac.uk/Output/1792860