Physics-based differentiable rendering is the task of estimating the derivatives of radiometric measures with respect to scene parameters. The ability to compute these derivatives is necessary for enabling gradient-based optimization in a diverse array of applications: from solving analysis-by-synthesis problems to training machine learning pipelines incorporating forward rendering processes. Unfortunately, physics-based differentiable rendering remains challenging, due to the complex and typically nonlinear relation between pixel intensities and scene parameters.

We introduce a differential theory of radiative transfer, which shows how individual components of the radiative transfer equation (RTE) can be differentiated with respect to arbitrary differentiable changes of a scene. Our theory encompasses the same generality as the standard RTE, allowing differentiation while accurately handling a large range of light transport phenomena such as volumetric absorption and scattering, anisotropic phase functions, and heterogeneity. To numerically estimate the derivatives given by our theory, we introduce an unbiased Monte Carlo estimator supporting arbitrary surface and volumetric configurations. Our technique differentiates path contributions symbolically and uses additional boundary integrals to capture geometric discontinuities such as visibility changes.

We validate our method by comparing our derivative estimations to those generated using the finite-difference method. Furthermore, we use a few synthetic examples inspired by real-world applications in inverse rendering, non-line-of-sight (NLOS) and biomedical imaging, and design, to demonstrate the practical usefulness of our technique.

• Paper: pdf (13 MB), latex (37 MB)
• Talk slides: pptx (9 MB)
• Supplemental document: pdf (5 MB)
• Supplemental material: html, zip (365 MB)
• Code: github

@article{Zhang:2019:DTRT,
  title={A Differential Theory of Radiative Transfer},
  author={Zhang, Cheng and Wu, Lifan and Zheng, Changxi and Gkioulekas, Ioannis and Ramamoorthi, Ravi and Zhao, Shuang},
  journal={ACM Trans. Graph.},
  volume={38},
  number={6},
  year={2019},
  pages={227:1--227:16}
}

We thank the anonymous reviewers for their constructive comments. We are grateful to Jerome Spanier for many helpful suggestions. This work was supported in part by NSF grants 1717178, 1730147, 1816041, 1900849, and 1900927, an NVIDIA fellowship, the Ronald L. Graham Chair, and the UC San Diego Center for Visual Computing.