Skip to main content

Research Repository

Advanced Search

Real-time Facial Animation for 3D Stylized Character with Emotion Dynamics

Pan, Ye; Zhang, Ruisi; Wang, Jingying; Ding, Yu; Mitchell, Kenny


Ye Pan

Ruisi Zhang

Jingying Wang

Yu Ding


Our aim is to improve animation production techniques' efficiency and effectiveness. We present two real-time solutions which drive character expressions in a geometrically consistent and perceptually valid way. Our first solution combines keyframe animation techniques with machine learning models. We propose a 3D emotion transfer network makes use of a 2D human image to generate a stylized 3D rig parameter. Our second solution combines blendshape-based motion capture animation techniques with machine learning models. We propose a blendshape adaption network which generates the character rig parameter motions with geometric consistency and temporally stability. We demonstrate the effectiveness of our system by comparing it to a commercial product Faceware. Results reveal that ratings of the recognition, intensity, and attractiveness of expressions depicted for animated characters via our systems are statistically higher than Faceware. Our results may be implemented into the animation pipeline, supporting animators to create expressions more rapidly and precisely.

Presentation Conference Type Conference Paper (Published)
Conference Name 31st ACM International Conference on Multimedia
Start Date Oct 29, 2023
End Date Nov 3, 2023
Online Publication Date Oct 27, 2023
Publication Date 2023-10
Deposit Date Nov 3, 2023
Publicly Available Date Oct 28, 2024
Publisher Association for Computing Machinery (ACM)
Pages 6851-6859
Book Title MM '23: Proceedings of the 31st ACM International Conference on Multimedia
ISBN 9798400701085
Keywords motion capture, virtual characters, emotion
Public URL
Related Public URLs


This file is under embargo until Oct 28, 2024 due to copyright reasons.

Contact to request a copy for personal use.

Related Outputs

You might also like

Downloadable Citations