Skip to main content

Research Repository

Advanced Search

All Outputs (76)

MienCap: Performance-based Facial Animation with Live Mood Dynamics (2022)
Presentation / Conference Contribution
Pan, Y., Zhang, R., Wang, J., Chen, N., Qiu, Y., Ding, Y., & Mitchell, K. (2022, March). MienCap: Performance-based Facial Animation with Live Mood Dynamics. Presented at 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Christchurch, New Zealand

Our purpose is to improve performance-based animation which can drive believable 3D stylized characters that are truly perceptual. By combining traditional blendshape animation techniques with machine learning models, we present a real time motion ca... Read More about MienCap: Performance-based Facial Animation with Live Mood Dynamics.

Collimated Whole Volume Light Scattering in Homogeneous Finite Media (2022)
Journal Article
Velinov, Z., & Mitchell, K. (2023). Collimated Whole Volume Light Scattering in Homogeneous Finite Media. IEEE Transactions on Visualization and Computer Graphics, 29(7), 3145-3157. https://doi.org/10.1109/TVCG.2021.3135764

Crepuscular rays form when light encounters an optically thick or opaque medium which masks out portions of the visible scene. Real-time applications commonly estimate this phenomena by connecting paths between light sources and the camera after a si... Read More about Collimated Whole Volume Light Scattering in Homogeneous Finite Media.

Embodied online dance learning objectives of CAROUSEL + (2021)
Presentation / Conference Contribution
Mitchell, K., Koniaris, B., Tamariz, M., Kennedy, J., Cheema, N., Mekler, E., Van Der Linden, P., Herrmann, E., Hämäläinen, P., McGregor, I., Slusallek, P., & Mac Williams, C. (2021, March). Embodied online dance learning objectives of CAROUSEL +. Presented at 2021 IEEE VR 6th Annual Workshop on K-12+ Embodied Learning through Virtual and Augmented Reality (KELVAR), Lisbon, Portugal

This is a position paper concerning the embodied dance learning objectives of the CAROUSEL + 1 project, which aims to impact how online immersive technologies influence multiuser interaction and communication with a focus on dancing and learning danc... Read More about Embodied online dance learning objectives of CAROUSEL +.

FaceMagic: Real-time Facial Detail Effects on Mobile (2020)
Presentation / Conference Contribution
Casas, L., Li, Y., & Mitchell, K. (2020). FaceMagic: Real-time Facial Detail Effects on Mobile. In SA '20: SIGGRAPH Asia 2020 Technical Communications (1-4). https://doi.org/10.1145/3410700.3425429

We present a novel real-time face detail reconstruction method capable of recovering high quality geometry on consumer mobile devices. Our system firstly uses a morphable model and semantic segmentation of facial parts to achieve robust self-calibrat... Read More about FaceMagic: Real-time Facial Detail Effects on Mobile.

Improving VIP viewer Gaze Estimation and Engagement Using Adaptive Dynamic Anamorphosis (2020)
Journal Article
Pan, Y., & Mitchell, K. (2021). Improving VIP viewer Gaze Estimation and Engagement Using Adaptive Dynamic Anamorphosis. International Journal of Human-Computer Studies, 147, Article 102563. https://doi.org/10.1016/j.ijhcs.2020.102563

Anamorphosis for 2D displays can provide viewer centric perspective viewing, enabling 3D appearance, eye contact and engagement, by adapting dynamically in real time to a single moving viewer’s viewpoint, but at the cost of distorted viewing for othe... Read More about Improving VIP viewer Gaze Estimation and Engagement Using Adaptive Dynamic Anamorphosis.

Active Learning for Interactive Audio-Animatronic Performance Design (2020)
Journal Article
Castellon, J., Bächer, M., McCrory, M., Ayala, A., Stolarz, J., & Mitchell, K. (2020). Active Learning for Interactive Audio-Animatronic Performance Design. The Journal of Computer Graphics Techniques, 9(3), 1-19

We present a practical neural computational approach for interactive design of Audio-Animatronic® facial performances. An offline quasi-static reference simulation, driven by a coupled mechanical assembly, accurately predicts hyperelastic skin deform... Read More about Active Learning for Interactive Audio-Animatronic Performance Design.

Props Alive: A Framework for Augmented Reality Stop Motion Animation (2020)
Presentation / Conference Contribution
Casas, L., Kosek, M., & Mitchell, K. (2017, March). Props Alive: A Framework for Augmented Reality Stop Motion Animation. Presented at 2017 IEEE 10th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS), Los Angeles, CA, USA

Stop motion animation evolved in the early days of cinema with the aim to create an illusion of movement with static puppets posed manually each frame. Current stop motion movies introduced 3D printing processes in order to acquire animations more ac... Read More about Props Alive: A Framework for Augmented Reality Stop Motion Animation.

PoseMMR: A Collaborative Mixed Reality Authoring Tool for Character Animation (2020)
Presentation / Conference Contribution
Pan, Y., & Mitchell, K. (2020, March). PoseMMR: A Collaborative Mixed Reality Authoring Tool for Character Animation. Presented at 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA

Augmented reality devices enable new approaches for character animation, e.g., given that character posing is three dimensional in nature it follows that interfaces with higher degrees-of-freedom (DoF) should outperform 2D interfaces. We present Pose... Read More about PoseMMR: A Collaborative Mixed Reality Authoring Tool for Character Animation.

Group-Based Expert Walkthroughs: How Immersive Technologies Can Facilitate the Collaborative Authoring of Character Animation (2020)
Presentation / Conference Contribution
Pan, Y., & Mitchell, K. (2020, March). Group-Based Expert Walkthroughs: How Immersive Technologies Can Facilitate the Collaborative Authoring of Character Animation. Presented at 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA

Immersive technologies have increasingly attracted the attention of the computer animation community in search of more intuitive and effective alternatives to the current sophisticated 2D interfaces. The higher affordances offered by 3D interaction,... Read More about Group-Based Expert Walkthroughs: How Immersive Technologies Can Facilitate the Collaborative Authoring of Character Animation.

Photo-Realistic Facial Details Synthesis from Single Image (2019)
Presentation / Conference Contribution
Chen, A., Chen, Z., Zhang, G., Zhang, Z., Mitchell, K., & Yu, J. (2019, October). Photo-Realistic Facial Details Synthesis from Single Image. Presented at 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea

We present a single-image 3D face synthesis technique that can handle challenging facial expressions while recovering fine geometric details. Our technique employs expression analysis for proxy face geometry generation and combines supervised and uns... Read More about Photo-Realistic Facial Details Synthesis from Single Image.

Recycling a Landmark Dataset for Real-time Facial Capture and Animation with Low Cost HMD Integrated Cameras (2019)
Presentation / Conference Contribution
Dos Santos Brito, C. J., & Mitchell, K. (2019, November). Recycling a Landmark Dataset for Real-time Facial Capture and Animation with Low Cost HMD Integrated Cameras. Presented at VRCAI ’19, Brisbane, QLD, Australia

Preparing datasets for use in the training of real-time face tracking algorithms for HMDs is costly. Manually annotated facial landmarks are accessible for regular photography datasets, but introspectively mounted cameras for VR face tracking have in... Read More about Recycling a Landmark Dataset for Real-time Facial Capture and Animation with Low Cost HMD Integrated Cameras.

JUNGLE: An Interactive Visual Platform for Collaborative Creation and Consumption of Nonlinear Transmedia Stories (2019)
Presentation / Conference Contribution
Kapadia, M., Muniz, C. M., Sohn, S. S., Pan, Y., Schriber, S., Mitchell, K., & Gross, M. (2019, November). JUNGLE: An Interactive Visual Platform for Collaborative Creation and Consumption of Nonlinear Transmedia Stories. Presented at International Conference on Interactive Digital Storytelling (ICIDS 2019), Salt Lake City, Utah, USA

JUNGLE is an interactive, visual platform for the collaborative manipulation and consumption of nonlinear transmedia stories. Intuitive visual interfaces encourage JUNGLE users to explore vast libraries of story worlds, expand existing stories, or co... Read More about JUNGLE: An Interactive Visual Platform for Collaborative Creation and Consumption of Nonlinear Transmedia Stories.

Depth codec for real-time, high-quality light field reconstruction (2019)
Patent
Mitchell, K., Koniaris, C., Kosek, M., & Sinclair, D. Depth codec for real-time, high-quality light field reconstruction. US20190313080A1

Systems, methods, and articles of manufacture are disclosed that enable the compression of depth data and real-time reconstruction of high-quality light fields. In one aspect, spatial compression and decompression of depth images is divided into the... Read More about Depth codec for real-time, high-quality light field reconstruction.

Light Field Synthesis Using Inexpensive Surveillance Camera Systems (2019)
Presentation / Conference Contribution
Dumbgen, F., Schroers, C., & Mitchell, K. (2019). Light Field Synthesis Using Inexpensive Surveillance Camera Systems. . https://doi.org/10.1109/icip.2019.8804269

We present a light field synthesis technique that achieves accurate reconstruction given a low-cost, wide-baseline camera rig. Our system integrates optical flow with methods for rectification, disparity estimation, and feature extraction, which we t... Read More about Light Field Synthesis Using Inexpensive Surveillance Camera Systems.

Intermediated Reality: A Framework for Communication Through Tele-Puppetry (2019)
Journal Article
Casas, L., & Mitchell, K. (2019). Intermediated Reality: A Framework for Communication Through Tele-Puppetry. Frontiers in Robotics and AI, 6, https://doi.org/10.3389/frobt.2019.00060

We introduce Intermediated Reality (IR), a framework for intermediated communication enabling collaboration through remote possession of entities (e.g., toys) that come to life in mobile Mediated Reality (MR). As part of a two-way conversation, each... Read More about Intermediated Reality: A Framework for Communication Through Tele-Puppetry.

Deep Precomputed Radiance Transfer for Deformable Objects (2019)
Presentation / Conference Contribution
Li, Y., Wiedemann, P., & Mitchell, K. (2019, May). Deep Precomputed Radiance Transfer for Deformable Objects. Presented at ACM Symposium on Interactive 3D Graphics and Games, Montreal, Quebec, Canada

We propose, DeepPRT, a deep convolutional neural network to compactly encapsulate the radiance transfer of a freely deformable object for rasterization in real-time. With pre-computation of radiance transfer (PRT) we can store complex light interac... Read More about Deep Precomputed Radiance Transfer for Deformable Objects.

Enhanced Shadow Retargeting with Light-Source Estimation Using Flat Fresnel Lenses (2019)
Journal Article
Casas, L., Fauconneau, M., Kosek, M., Mclister, K., & Mitchell, K. (2019). Enhanced Shadow Retargeting with Light-Source Estimation Using Flat Fresnel Lenses. Computers, 8(2), Article 29. https://doi.org/10.3390/computers8020029

Shadow-retargeting maps depict the appearance of real shadows to virtual shadows given corresponding deformation of scene geometry, such that appearance is seamlessly maintained. By performing virtual shadow reconstruction from unoccluded real-shadow... Read More about Enhanced Shadow Retargeting with Light-Source Estimation Using Flat Fresnel Lenses.

Memory Allocation For Seamless Media Content Presentation (2019)
Patent
Mitchell, K., Koniaris, C., & Chitalu, F. (2019). Memory Allocation For Seamless Media Content Presentation. US20190096028

A system for performing memory allocation for seamless media content presentation includes a computing platform having a CPU, a GPU having a GPU memory, and a main memory storing a memory allocation software code. The CPU executes the memory allocati... Read More about Memory Allocation For Seamless Media Content Presentation.

Feature-preserving detailed 3D face reconstruction from a single image (2018)
Presentation / Conference Contribution
Li, Y., Ma, L., Fan, H., & Mitchell, K. (2018, December). Feature-preserving detailed 3D face reconstruction from a single image. Presented at the 15th ACM SIGGRAPH European Conference, London, United Kingdom

Dense 3D face reconstruction plays a fundamental role in visual media production involving digital actors. We improve upon high fidelity reconstruction from a single 2D photo with a reconstruction framework that is robust to large variations in expre... Read More about Feature-preserving detailed 3D face reconstruction from a single image.