Ye Pan
Expressive Talking Avatars
Pan, Ye; Tan, Shuai; Cheng, Shengran; Lin, Qunfen; Zeng, Zijiao; Mitchell, Kenny
Authors
Shuai Tan
Shengran Cheng
Qunfen Lin
Zijiao Zeng
Prof Kenny Mitchell K.Mitchell2@napier.ac.uk
Professor
Abstract
Stylized avatars are common virtual representations used in VR to support interaction and communication between remote collaborators. However, explicit expressions are notoriously difficult to create, mainly because most current methods rely on geometric markers and features modeled for human faces, not stylized avatar faces. To cope with the challenge of emotional and expressive generating talking avatars, we build the Emotional Talking Avatar Dataset which is a talking-face video corpus featuring 6 different stylized characters talking with 7 different emotions. Together with the dataset, we also release an emotional talking avatar generation method which enables the manipulation of emotion. We validated the effectiveness of our dataset and our method in generating audio based puppetry examples, including comparisons to state-of-the-art techniques and a user study. Finally, various applications of this method are discussed in the context of animating avatars in VR.
Citation
Pan, Y., Tan, S., Cheng, S., Lin, Q., Zeng, Z., & Mitchell, K. (2024). Expressive Talking Avatars. IEEE Transactions on Visualization and Computer Graphics, 30(5), 2538-2548. https://doi.org/10.1109/TVCG.2024.3372047
Journal Article Type | Article |
---|---|
Conference Name | IEEE VR 2024 |
Acceptance Date | Dec 17, 2023 |
Online Publication Date | Mar 4, 2024 |
Publication Date | 2024 |
Deposit Date | Dec 18, 2023 |
Publicly Available Date | Mar 4, 2024 |
Print ISSN | 1077-2626 |
Electronic ISSN | 1941-0506 |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 30 |
Issue | 5 |
Pages | 2538-2548 |
DOI | https://doi.org/10.1109/TVCG.2024.3372047 |
Keywords | Human-centered computing, Human computer interaction (HCI), HCI design and evaluation methods, User studies, Human-centered computing, Computer graphics, Graphics systems and interfaces, Virtual reality |
Public URL | http://researchrepository.napier.ac.uk/Output/3433443 |
Related Public URLs | https://ieeevr.org/2024/ |
Files
Emotional Talking Avatars
(9.7 Mb)
PDF
You might also like
Iterative image warping
(2012)
Journal Article
OSCAM-optimized stereoscopic camera control for interactive 3D
(2011)
Journal Article
Capture and analysis of racing gameplay metrics
(2011)
Journal Article
OSCAM - Optimized Stereoscopic Camera Control for Interactive 3D
(2011)
Journal Article
Integrating real-time fluid simulation with a voxel engine
(2016)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search