Real-time Facial Animation for 3D Stylized Character with Emotion Dynamics

Pan, Ye; Zhang, Ruisi; Wang, Jingying; Ding, Yu; Mitchell, Kenny

Abstract

Our aim is to improve animation production techniques' efficiency and effectiveness. We present two real-time solutions which drive character expressions in a geometrically consistent and perceptually valid way. Our first solution combines keyframe animation techniques with machine learning models. We propose a 3D emotion transfer network makes use of a 2D human image to generate a stylized 3D rig parameter. Our second solution combines blendshape-based motion capture animation techniques with machine learning models. We propose a blendshape adaption network which generates the character rig parameter motions with geometric consistency and temporally stability. We demonstrate the effectiveness of our system by comparing it to a commercial product Faceware. Results reveal that ratings of the recognition, intensity, and attractiveness of expressions depicted for animated characters via our systems are statistically higher than Faceware. Our results may be implemented into the animation pipeline, supporting animators to create expressions more rapidly and precisely.

Citation

Pan, Y., Zhang, R., Wang, J., Ding, Y., & Mitchell, K. (2023, October). Real-time Facial Animation for 3D Stylized Character with Emotion Dynamics. Presented at 31st ACM International Conference on Multimedia, Ottawa, Canada

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 31st ACM International Conference on Multimedia
• Start Date: Oct 29, 2023
• End Date: Nov 3, 2023
• Online Publication Date: Oct 27, 2023
• Publication Date: 2023-10
• Deposit Date: Nov 3, 2023
• Publicly Available Date: Oct 28, 2024
• Publisher: Association for Computing Machinery (ACM)
• Pages: 6851-6859
• Book Title: MM '23: Proceedings of the 31st ACM International Conference on Multimedia
• ISBN: 9798400701085
• DOI: https://doi.org/10.1145/3581783.3613803
• Keywords: motion capture, virtual characters, emotion
• Public URL: http://researchrepository.napier.ac.uk/Output/3367808
• Related Public URLs: https://www.acmmm2023.org/acm-mm-2023/