Synthetic Prior Design for Real-Time Face Tracking

McDonagh, Steven; Klaudiny, Martin; Bradley, Derek; Beeler, Thabo; Matthews, Iain; Mitchell, Kenny

Abstract

Real-time facial performance capture has recently been gaining popularity in virtual film production, driven by advances in machine learning, which allows for fast inference of facial geometry from video streams. These learning-based approaches are significantly influenced by the quality and amount of labelled training data. Tedious construction of training sets from real imagery can be replaced by rendering a facial animation rig under on-set conditions expected at runtime. We learn a synthetic actor-specific prior by adapting a state-of-the-art facial tracking method. Synthetic training significantly reduces the capture and annotation burden and in theory allows generation of an arbitrary amount of data. But practical realities such as training time and compute resources still limit the size of any training set. We construct better and smaller training sets by investigating which facial image appearances are crucial for tracking accuracy, covering the dimensions of expression, viewpoint and illumination. A reduction of training data in 1-2 orders of magnitude is demonstrated whilst tracking accuracy is retained for challenging on-set footage.

Citation

McDonagh, S., Klaudiny, M., Bradley, D., Beeler, T., Matthews, I., & Mitchell, K. (2016, October). Synthetic Prior Design for Real-Time Face Tracking. Presented at 2016 Fourth International Conference on 3D Vision (3DV)

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2016 Fourth International Conference on 3D Vision (3DV)
• Start Date: Oct 25, 2016
• End Date: Oct 28, 2016
• Acceptance Date: Oct 1, 2016
• Online Publication Date: Dec 19, 2016
• Publication Date: Dec 19, 2016
• Deposit Date: Dec 12, 2017
• Publicly Available Date: Dec 13, 2017
• Publisher: Institute of Electrical and Electronics Engineers
• Book Title: 2016 Fourth International Conference on 3D Vision (3DV),
• ISBN: 9781509054077
• DOI: https://doi.org/10.1109/3dv.2016.72
• Keywords: real-time tracking, face performance capture,
• Public URL: http://researchrepository.napier.ac.uk/Output/951441
• Contract Date: Dec 12, 2017