Skip to main content

Research Repository

Advanced Search

FaceMagic: Real-time Facial Detail Effects on Mobile

Casas, Llogari; Li, Yue; Mitchell, Kenny


Yue Li


We present a novel real-time face detail reconstruction method capable of recovering high quality geometry on consumer mobile devices. Our system firstly uses a morphable model and semantic segmentation of facial parts to achieve robust self-calibration. We then capture fine-scale surface details using a patch-based Shape from Shading (SfS) approach. We pre-compute the patch-wise constant Moore–Penrose inverse matrix of the resulting linear system to achieve real-time performance. Our method achieves high interactive frame-rates and experiments show that our new approach is capable of reconstructing high-fidelity geometry with corresponding results to off-line techniques. We illustrate this through comparisons with off-line and on-line related works, and include demonstrations of novel face detail shader effects processing.

Presentation Conference Type Conference Paper (Published)
Conference Name SA '20: SIGGRAPH Asia 2020
Start Date Dec 6, 2020
End Date Dec 11, 2020
Acceptance Date Sep 16, 2020
Online Publication Date Nov 17, 2020
Publication Date 2020-12
Deposit Date Dec 4, 2020
Publicly Available Date Dec 8, 2020
Publisher Association for Computing Machinery (ACM)
Pages 1-4
Book Title SA '20: SIGGRAPH Asia 2020 Technical Communications
ISBN 9781450380805
Keywords Augmented Reality
Public URL
Publisher URL


FaceMagic: Real-time Facial Detail Effects On Mobile (accepted version) (25.5 Mb)

You might also like

Downloadable Citations