Path-Space Differentiable Rendering of Participating Media
Cheng Zhang*, Zihan Yu*, and Shuang Zhao (*: equal contribution)
University of California, Irvine
ACM Transactions on Graphics (SIGGRAPH 2021), 40(4), 2021
teaser
Abstract

Physics-based differentiable rendering---which focuses on estimating derivatives of radiometric detector responses with respect to arbitrary scene parameters---has a diverse array of applications from solving analysis-by-synthesis problems to training machine-learning pipelines incorporating forward-rendering processes. Unfortunately, existing general-purpose differentiable rendering techniques lack either the generality to handle volumetric light transport or the flexibility to devise Monte Carlo estimators capable of handling complex geometries and light transport effects.

In this paper, we bridge this gap by showing how generalized path integrals can be differentiated with respect to arbitrary scene parameters. Specifically, we establish the mathematical formulation of generalized differential path integrals that capture both interfacial and volumetric light transport. Our formulation allows the development of advanced differentiable rendering algorithms capable of efficiently handling challenging geometric discontinuities and light transport phenomena such as volumetric caustics.

We validate our method by comparing our derivative estimates to those generated using the finite differences. Further, to demonstrate the effectiveness of our technique, we compare both differentiable rendering and inverse rendering performance with state-of-the-art methods.

Downloads
  • Paper: pdf (6.4 MB)
  • Supplemental material: html, zip (80 MB)
  • Code: coming soon!
Bibtex citation
@article{Zhang:2021:PSDR,
  title={Path-Space Differentiable Rendering of Participating Media},
  author={Zhang, Cheng and Yu, Zihan and Zhao, Shuang},
  journal={ACM Trans. Graph.},
  volume={40},
  number={4},
  year={2021},
  pages={76:1--76:15}
}
Acknowledgments

We thank the anonymous reviewers for their constructive comments and suggestions. This work was partially supported by NSF grant 1900927.