MoodFlow: Orchestrating Conversations with Emotionally Intelligent Avatars in Mixed Reality

Casas, Llogari; Hannah, Samantha; Mitchell, Kenny

Abstract

MoodFlow presents a novel approach at the intersection of mixed reality and conversational artificial intelligence for emotionally intelligent avatars. Through a state machine embedded in user prompts, the system decodes emotional nuances, enabling avatars to respond authentically to the spectrum of human emotions. Our system employs expressive avatars with a shared structure, allowing for seamless animation transferability between avatars with distinct outlook. The avatars, optimized for mixed reality, incorporate low-poly designs and toon shader stylization. This immersive journey transforms virtual conversations into open-ended dialogues, where avatars go beyond scripted interactions, adapting in real-time based on emotional context. Beyond entertainment, the approach envisions diverse applications, including virtual therapy, education, entertainment, corporate communication, and social interactions by opening doors to emotionally rich experiences across sectors.

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: ANIVAE 2024 : 7th IEEE VR Internal Workshop on Animation in Virtual and Augmented Environments
• Start Date: Mar 16, 2024
• End Date: Mar 21, 2024
• Acceptance Date: Jan 19, 2024
• Deposit Date: Mar 16, 2024
• Publisher: Institute of Electrical and Electronics Engineers
• Keywords: User/Machine Systems, Human-centered computing, Three-Dimensional Graphics and Realism, Animation, Methodology and Techniques, Games
• Public URL: http://researchrepository.napier.ac.uk/Output/3567584
• Publisher URL: https://www.computer.org/csdl/proceedings/1836626
• Related Public URLs: https://anivae.fhstp.ac.at/; https://ieeevr.org/2024/