Skip to main content

Research Repository

Advanced Search

Rapid one-shot acquisition of dynamic VR avatars

Malleson, Charles; Kosek, Maggie; Klaudiny, Martin; Huerta, Ivan; Bazin, Jean-Charles; Sorkine-Hornung, Alexander; Mine, Mark; Mitchell, Kenny

Authors

Charles Malleson

Maggie Kosek

Martin Klaudiny

Ivan Huerta

Jean-Charles Bazin

Alexander Sorkine-Hornung

Mark Mine



Abstract

We present a system for rapid acquisition of bespoke, animatable, full-body avatars including face texture and shape. A blendshape rig with a skeleton is used as a template for customization. Identity blendshapes are used to customize the body and face shape at the fitting stage, while animation blendshapes allow the face to be animated. The subject assumes a T-pose and a single snapshot is captured using a stereo RGB plus depth sensor rig. Our system automatically aligns a photo texture and fits the 3D shape of the face. The body shape is stylized according to body dimensions estimated from segmented depth. The face identity blendweights are optimised according to image-based facial landmarks, while a custom texture map for the face is generated by warping the input images to a reference texture according to the facial landmarks. The total capture and processing time is under 10 seconds and the output is a light-weight, game-engine-ready avatar which is recognizable as the subject. We demonstrate our system in a VR environment in which each user sees the other users' animated avatars through a VR headset with real-time audio-based facial animation and live body motion tracking, affording an enhanced level of presence and social engagement compared to generic avatars.

Presentation Conference Type Conference Paper (Published)
Conference Name 2017 IEEE Virtual Reality (VR)
Start Date Mar 18, 2017
End Date Mar 22, 2017
Acceptance Date Mar 1, 2017
Online Publication Date Apr 6, 2017
Publication Date Apr 6, 2017
Deposit Date Sep 12, 2017
Publisher Institute of Electrical and Electronics Engineers
Series ISSN 2375-5334
Book Title IEEE 2017 Virtual Reality (VR)
ISBN 9781509066476
DOI https://doi.org/10.1109/vr.2017.7892240
Keywords Avatars, capture, virtual reality
Public URL http://researchrepository.napier.ac.uk/Output/951398