Skip to main content

Research Repository

Advanced Search

COG-MHEAR: Towards cognitively-inspired, 5G-IoT enabled multi-modal hearing aids

People Involved

Project Description

Embracing the multimodal nature of speech presents both opportunities and challenges for hearing assistive technology:
on the one hand there are opportunities for the design of new multimodal audio-visual (AV) algorithms; on the other hand,
multimodality challenges the current standards for hearing aid evaluation, which generally considers the perception of the
audio signal in insolation.
Our hypothesis is that it is possible to contextually combine visual and acoustic inputs to produce a "real-time" cognitively inspired, multimodal hearing device that is able to significantly boost speech intelligibility in the everyday listening
environments in which traditional audio-only hearing devices prove ineffective. To test this hypothesis, and build on recent
preliminary research by the team, into off-line development of lip-reading and deep learning-driven AV speech
enhancement (SE) algorithms, we aim to ambitiously develop and clinically validate next-generation, cognitively-inspired,
AV hearing-aid (HA) prototypes, operating in real-time through 'off- and on-chip' implementations.
The disruptive device will autonomously and contextually adapt to the nature and quality of its visual and acoustic
environmental inputs. We will achieve this aim by the design of a transformative, privacy-preserving AV SE framework,
integrating a next-generation communication network solution: 5G Cloud-Radio Access Network (C-RAN), with the Internet
of Things (IoT), context-aware machine learning and strong privacy algorithms, for optimised, 'off-' and 'on-chip' real-time
processing.

Type of Project P03 - Research Councils
Status Project Live
Funder(s) Engineering and Physical Sciences Research Council
Value £3,258,999.00
Project Dates Mar 1, 2021 - Feb 28, 2026
Partner Organisations Health and Social Care Alliance Scotland
Royal National Institute for Deaf People
Nokia
Alpha Data Parallel Systems Ltd
University College London
Digital Health Institute
NHS Lothian
The Data Lab
Sonova AG



You might also like

FOCAS

FOCAS Jan 1, 2013 - Feb 28, 2016
FOCAS is a coordination action in the area of collective adaptive systems. It provides increased visibility to the research carried out by projects funded by the FOCAS FET Proactive Initiative and others in research fields related to collective adapt... Read More about FOCAS.

Project Quaisten Jun 1, 2014 - Aug 1, 2015
To develop a question generator API to pull information from the web, based on defined questions types, confirming correct answers and implementing a process of question difficulty based on metrics about the individual question type and possible answ... Read More about Project Quaisten.

Fragment Finder Mar 27, 2015 - Jan 18, 2016
Fragment Finder (FF) enables a new, high-speed approach to digital forensics. It is unique in that it will build a more efficient technical architecture for the creation, storage and use of hash signatures in digital forensics. The key focus of FF is... Read More about Fragment Finder.