Skip to main content

Research Repository

Advanced Search

COG-MHEAR: Towards cognitively-inspired, 5G-IoT enabled multi-modal hearing aids

People Involved

Project Description

Embracing the multimodal nature of speech presents both opportunities and challenges for hearing assistive technology:
on the one hand there are opportunities for the design of new multimodal audio-visual (AV) algorithms; on the other hand,
multimodality challenges the current standards for hearing aid evaluation, which generally considers the perception of the
audio signal in insolation.
Our hypothesis is that it is possible to contextually combine visual and acoustic inputs to produce a "real-time" cognitively inspired, multimodal hearing device that is able to significantly boost speech intelligibility in the everyday listening
environments in which traditional audio-only hearing devices prove ineffective. To test this hypothesis, and build on recent
preliminary research by the team, into off-line development of lip-reading and deep learning-driven AV speech
enhancement (SE) algorithms, we aim to ambitiously develop and clinically validate next-generation, cognitively-inspired,
AV hearing-aid (HA) prototypes, operating in real-time through 'off- and on-chip' implementations.
The disruptive device will autonomously and contextually adapt to the nature and quality of its visual and acoustic
environmental inputs. We will achieve this aim by the design of a transformative, privacy-preserving AV SE framework,
integrating a next-generation communication network solution: 5G Cloud-Radio Access Network (C-RAN), with the Internet
of Things (IoT), context-aware machine learning and strong privacy algorithms, for optimised, 'off-' and 'on-chip' real-time
processing.

Status Project Live
Funder(s) Engineering and Physical Sciences Research Council
Value £3,258,999.00
Project Dates Mar 1, 2021 - Feb 28, 2026
Partner Organisations Health and Social Care Alliance Scotland
Royal National Institute for Deaf People
Nokia
Alpha Data Parallel Systems Ltd
University College London
Digital Health Institute
NHS Lothian
The Data Lab
Sonova AG



You might also like

Artificial Intelligence (AI)-powered dashboard for COVID-19 related public sentiment and opinion mining in social media platforms May 1, 2020 - Nov 17, 2020
: The project will aid in understanding and mitigating the direct and indirect impacts of the COVID-19 pandemic, by creating an AI-driven dashboard for policymakers, public health and clinical practitioners. This will enable continuous monitoring, pr... Read More about Artificial Intelligence (AI)-powered dashboard for COVID-19 related public sentiment and opinion mining in social media platforms.

KTP Ace Aquatec Jun 1, 2021 - Nov 30, 2023
To develop, test and incorporate Innovative AI based approaches to improve accuracy of Individual fish identification and sea lice detection.