MienCap: Performance-based Facial Animation with Live Mood Dynamics

Pan, Ye; Zhang, Ruisi; Wang, Jingying; Chen, Nengfu; Qiu, Yilin; Ding, Yu; Mitchell, Kenny

Abstract

Our purpose is to improve performance-based animation which can drive believable 3D stylized characters that are truly perceptual. By combining traditional blendshape animation techniques with machine learning models, we present a real time motion capture system, called MienCap, which drive character expressions in a geometrically consistent and perceptually valid way. We demon-strate the effectiveness of our system by comparing to a commercial product Faceware. Results reveal that ratings of the recognition of expressions depicted for animated characters via our systems are statistically higher than Faceware. Our results may be implemented into the VR filmmaking and animation pipeline, and provide animators with a system for creating the expressions they wish to use more quickly and accurately.

Citation

Pan, Y., Zhang, R., Wang, J., Chen, N., Qiu, Y., Ding, Y., & Mitchell, K. (2022). MienCap: Performance-based Facial Animation with Live Mood Dynamics. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (645-646). https://doi.org/10.1109/vrw55335.2022.00178

• Conference Name: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
• Conference Location: Christchurch, New Zealand
• Start Date: Mar 12, 2022
• End Date: Mar 16, 2022
• Online Publication Date: Apr 20, 2022
• Publication Date: 2022-03
• Deposit Date: Jul 15, 2022
• Publisher: Institute of Electrical and Electronics Engineers
• Pages: 645-646
• Book Title: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
• DOI: https://doi.org/10.1109/vrw55335.2022.00178
• Public URL: http://researchrepository.napier.ac.uk/Output/2889043