Deep Precomputed Radiance Transfer for Deformable Objects

Li, Yue; Wiedemann, Pablo; Mitchell, Kenny

Abstract

We propose, DeepPRT, a deep convolutional neural network to compactly encapsulate the radiance transfer of a freely deformable object for rasterization in real-time.
With pre-computation of radiance transfer (PRT) we can store complex light interactions appropriate to the shape of a given object at each surface point for subsequent real-time rendering via fast linear algebra evaluation against the viewing direction and distant light environment. However, performing light transport projection into an efficient basis representation, such as Spherical Harmonics (SH), requires a numerical Monte Carlo integration computation, limiting usage to rigid only objects or highly constrained deformation sequences. The bottleneck, when considering freely deformable objects, is the heavy memory requirement to wield all pre-computations in rendering with global illumination results.
We present a compact representation of PRT for deformable objects with fixed memory consumption, which solves diverse non-linear deformations and is shown to be effective beyond the input training set. Specifically, a U-Net is trained to predict the coefficients of the transfer function (SH coefficients in this case), for a given animation's shape query each frame in real-time.
We contribute deep learning of PRT within a parametric surface space representation via geometry images using harmonic mapping with a texture space filling energy minimization variant. This surface representation facilitates the learning procedure, removing irrelevant, deformation invariant information; and supports standard convolution operations. Finally, comparisons with ground truth and a recent linear morphable-model method is provided.

Citation

Li, Y., Wiedemann, P., & Mitchell, K. (2019, May). Deep Precomputed Radiance Transfer for Deformable Objects. Presented at ACM Symposium on Interactive 3D Graphics and Games, Montreal, Quebec, Canada

• Presentation Conference Type: Conference Paper (published)
• Conference Name: ACM Symposium on Interactive 3D Graphics and Games
• Start Date: May 21, 2019
• End Date: May 23, 2019
• Acceptance Date: Feb 25, 2019
• Online Publication Date: Apr 2, 2019
• Publication Date: May 15, 2019
• Deposit Date: Apr 2, 2019
• Publicly Available Date: Apr 3, 2019
• Journal: Proceedings of the ACM on Computer Graphics and Interactive Techniques
• Print ISSN: 2577-6193
• Electronic ISSN: 2577-6193
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Volume: 2
• Issue: 1
• Article Number: 3
• Series ISSN: 2577-6193
• DOI: https://doi.org/10.1145/3320284
• Keywords: Real-time rendering, global illumination, pre-computed radiance transfer, spherical harmonics, deep learning
• Public URL: http://researchrepository.napier.ac.uk/Output/1702389
• Publisher URL: https://dl.acm.org/pub.cfm?id=J1615
• Contract Date: Apr 2, 2019