Skip to main content

Research Repository

Advanced Search

MoodFlow: Orchestrating Conversations with Emotionally Intelligent Avatars in Mixed Reality

Casas, Llogari; Hannah, Samantha; Mitchell, Kenny

Authors

Llogari Casas

Samantha Hannah



Abstract

MoodFlow presents a novel approach at the intersection of mixed reality and conversational artificial intelligence for emotionally intelligent avatars. Through a state machine embedded in user prompts, the system decodes emotional nuances, enabling avatars to respond authentically to the spectrum of human emotions. Our system employs expressive avatars with a shared structure, allowing for seamless animation transferability between avatars with distinct outlook. The avatars, optimized for mixed reality, incorporate low-poly designs and toon shader stylization. This immersive journey transforms virtual conversations into open-ended dialogues, where avatars go beyond scripted interactions, adapting in real-time based on emotional context. Beyond entertainment, the approach envisions diverse applications, including virtual therapy, education, entertainment, corporate communication, and social interactions by opening doors to emotionally rich experiences across sectors.

Citation

Casas, L., Hannah, S., & Mitchell, K. (2024, March). MoodFlow: Orchestrating Conversations with Emotionally Intelligent Avatars in Mixed Reality. Presented at ANIVAE 2024 : 7th IEEE VR Internal Workshop on Animation in Virtual and Augmented Environments, Orlando, Florida

Presentation Conference Type Conference Paper (published)
Conference Name ANIVAE 2024 : 7th IEEE VR Internal Workshop on Animation in Virtual and Augmented Environments
Start Date Mar 16, 2024
End Date Mar 21, 2024
Acceptance Date Jan 19, 2024
Online Publication Date May 29, 2024
Publication Date 2024
Deposit Date Mar 16, 2024
Publicly Available Date May 29, 2024
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Book Title 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
DOI https://doi.org/10.1109/VRW62533.2024.00021
Keywords User/Machine Systems, Human-centered computing, Three-Dimensional Graphics and Realism, Animation, Methodology and Techniques, Games
Public URL http://researchrepository.napier.ac.uk/Output/3567584
Publisher URL https://www.computer.org/csdl/proceedings/1836626
Related Public URLs https://anivae.fhstp.ac.at/
https://ieeevr.org/2024/

Files

MoodFlow: Orchestrating Conversations with Emotionally Intelligent Avatars in Mixed Reality (accepted version) (12.6 Mb)
PDF

Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/

Copyright Statement
IEEE grants the application of a CC BY license on the Accepted Manuscript, not the Final Published version, of the article. No delay or embargo period is required [...] as long as the funder is properly identified.






You might also like



Downloadable Citations