Differentiable Time-Gated Rendering

Supplemental Materials

Lifan Wu1*, Guangyan Cai2*, Ravi Ramamoorthi3, and Shuang Zhao2
1NVIDIA        2University of California, Irvine        3University of California, San Diego        (* equal contribution)

In this material, we show several synthetic inverse-rendering results. For each example, we minimize the image loss (measured in RMSE) between input (steady-state or ToF) target images and renderings. Additionally, we use parameter RMSE to evaluate the quality of inverse-rendering results. This information is not used by the optimizations.

Left click the images below to start/pause; right click to reset the animations.
Inverse-Rendering Comparisons

In what follows, we demonstrate the necessity of supporting global interreflection and the practical advantage of our method over finite differences by comparing inverse-rendering performances.

This Corridor scene includes a two-segment corridor with the camera and an area light located at the opening of the corridor. Taking as input 20 target ToF images, we search for global translations in x- and z-directions of the four vertices at the end of the second segment.

We use the Adam method to solve the inverse-rendering optimizations with identical initial configurations and learning rates for all methods. We also adjust the sample count so that each iteration takes approximately equal time for all methods.

3-bounce interreflection

When limiting the maximal number of reflections to three, the resulting time-gated renderings suffer high energy loss, causing the optimization to diverge.

Full interreflection (finite differences)

When using full interreflections with derivatives computed with finite differences, the forward renderings are accurate. Unfortunately, the optimization still fails to find the correct solution due to the bias and high variance in FD gradient estimates.

Full interreflection (ours)

Using full interreflections and derivatives estimated using our method, the inverse-rendering optimization successfully finds the correct result.


We now demonstrate the effectiveness of our ellipsoidal next-event estimation and antithetic sampling for handling near-delta path importance functions.

This Cube scene contains a glossy cube lit by an area light. We search for the vertical position of the cube using four target ToF images with near-delta path importance functions.

We use identical initial configurations, learning rates, and time per iteration for all methods.


Without ellipsoidal NEE or antithetic sampling, the standard estimator suffers from very high variance, causing the optimization to diverge.

Ellipsoidal NEE + antithetic sampling

With our ellipsoidal NEE and antithetic sampling, significant variance reduction can be achieved, allowing the optimization to converge easily.

Height field

We now compare inverse-rendering performance using steady-state and time-gated images. In this example, we have a diffuse surface lit and viewed from above. We search for the vertical positions of the 121 vertices of this surface. We use the Adam method with identical initial configurations to solve both optimizations.


When using one steady-state target image, the problem becomes highly under-constrained, causing the inverse-rendering optimization to get stuck at a local minimum.


When using 12 time-gated images (from the same camera location), the added information allows the optimization to converge at the correct solution.

Inverse-Rendering Results