Intermediated Reality with an AI 3D Printed Character

Casas, Llogari; Mitchell, Kenny

Abstract

We introduce live character conversational interactions in Intermediated Reality to bring real-world objects to life in Augmented Reality (AR) and Artificial Intelligence (AI). The AI recognizes live speech and generates short character responses, syncing the character’s facial expressions with speech audio. The Intermediated Reality AR warping technique allows for a high degree of realism in the animated facial expressions and movements reusing live video optical appearance direct from the device’s embedded camera. The proposed applications of Intermediated Reality with AI are exemplified through the captivating fusion of these technologies in toy interactive storytelling broadcasts and social telepresence scenarios. This innovative combination allows individuals to engage with AI characters in a natural and intuitive manner, creating new opportunities for social engagement and entertainment.

Citation

Casas, L., & Mitchell, K. (2023). Intermediated Reality with an AI 3D Printed Character. In SIGGRAPH '23: ACM SIGGRAPH 2023 Real-Time Live!. https://doi.org/10.1145/3588430.3597251

• Conference Name: ACM SIGGRAPH 2023 Real-Time Live!
• Conference Location: Los Angeles, CA, USA
• Start Date: Aug 6, 2023
• End Date: Aug 10, 2023
• Acceptance Date: May 10, 2023
• Online Publication Date: Jul 23, 2023
• Publication Date: 2023-07
• Deposit Date: Aug 4, 2023
• Publicly Available Date: Aug 7, 2023
• Publisher: Association for Computing Machinery (ACM)
• Book Title: SIGGRAPH '23: ACM SIGGRAPH 2023 Real-Time Live!
• ISBN: 9798400701580
• DOI: https://doi.org/10.1145/3588430.3597251
• Public URL: http://researchrepository.napier.ac.uk/Output/3148953