Charles Malleson
Rapid one-shot acquisition of dynamic VR avatars
Malleson, Charles; Kosek, Maggie; Klaudiny, Martin; Huerta, Ivan; Bazin, Jean-Charles; Sorkine-Hornung, Alexander; Mine, Mark; Mitchell, Kenny
Authors
Maggie Kosek
Martin Klaudiny
Ivan Huerta
Jean-Charles Bazin
Alexander Sorkine-Hornung
Mark Mine
Prof Kenny Mitchell K.Mitchell2@napier.ac.uk
Professor
Abstract
We present a system for rapid acquisition of bespoke, animatable, full-body avatars including face texture and shape. A blendshape rig with a skeleton is used as a template for customization. Identity blendshapes are used to customize the body and face shape at the fitting stage, while animation blendshapes allow the face to be animated. The subject assumes a T-pose and a single snapshot is captured using a stereo RGB plus depth sensor rig. Our system automatically aligns a photo texture and fits the 3D shape of the face. The body shape is stylized according to body dimensions estimated from segmented depth. The face identity blendweights are optimised according to image-based facial landmarks, while a custom texture map for the face is generated by warping the input images to a reference texture according to the facial landmarks. The total capture and processing time is under 10 seconds and the output is a light-weight, game-engine-ready avatar which is recognizable as the subject. We demonstrate our system in a VR environment in which each user sees the other users' animated avatars through a VR headset with real-time audio-based facial animation and live body motion tracking, affording an enhanced level of presence and social engagement compared to generic avatars.
Citation
Malleson, C., Kosek, M., Klaudiny, M., Huerta, I., Bazin, J., Sorkine-Hornung, A., …Mitchell, K. (2017). Rapid one-shot acquisition of dynamic VR avatars. In IEEE 2017 Virtual Reality (VR). https://doi.org/10.1109/vr.2017.7892240
Conference Name | 2017 IEEE Virtual Reality (VR) |
---|---|
Conference Location | Los Angeles, US |
Start Date | Mar 18, 2017 |
End Date | Mar 22, 2017 |
Acceptance Date | Mar 1, 2017 |
Online Publication Date | Apr 6, 2017 |
Publication Date | Apr 6, 2017 |
Deposit Date | Sep 12, 2017 |
Publisher | Institute of Electrical and Electronics Engineers |
Series ISSN | 2375-5334 |
Book Title | IEEE 2017 Virtual Reality (VR) |
ISBN | 9781509066476 |
DOI | https://doi.org/10.1109/vr.2017.7892240 |
Keywords | Avatars, capture, virtual reality |
Public URL | http://researchrepository.napier.ac.uk/Output/951398 |
You might also like
Props Alive: A Framework for Augmented Reality Stop Motion Animation
(2020)
Conference Proceeding
IRIDiuM+: deep media storytelling with non-linear light field video
(2017)
Conference Proceeding
Real-time rendering with compressed animated light fields.
(2017)
Conference Proceeding
IRIDiuM: immersive rendered interactive deep media
(2016)
Conference Proceeding
Stereohaptics: a haptic interaction toolkit for tangible virtual experiences
(2016)
Conference Proceeding
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search