Physics-Based Differentiable Rendering

A Comprehensive Introduction

A Comprehensive Introduction

Shuang Zhao^{1},
Wenzel Jakob^{2},
and Tzu-Mao Li^{3}

^{1}University of California, Irvine
^{2}EPFL
^{3}MIT CSAIL

SIGGRAPH 2020 Course

Abstract

Physics-based rendering algorithms generate photorealistic images by simulating the flow of light through a detailed mathematical representation of a virtual scene. In contrast, **physics-based differentiable rendering** algorithms focus on computing derivative of images exhibiting complex light transport effects (e.g., soft shadows, interreflection, and caustics) with respect to arbitrary scene parameters such as camera pose, object geometry (e.g., vertex positions) as well as spatially varying material properties expressed as 2D textures and 3D volumes. This new level of generality has made physics-based differentiable rendering a key ingredient for solving many challenging inverse-rendering problems, that is, the search of scene configurations optimizing user-specified objective functions, using gradient-based methods

Further, these techniques can be incorporated into probabilistic inference and machine learning pipelines. For instance, differentiable renderers allow "rendering losses" to be computed with complex light transport effects captured. Additionally, they can be used as generative models that synthesize photorealistic images.

__ Challenges.__ Compared to its "ordinary" counterpart, physics-based differentiable rendering introduces unique theoretical and practical challenges. For instance, practical problems can involve many (e.g., 10

Course Recording

Resources

© 2020. All Rights Reserved.