Efficient Estimation of Boundary Integrals for Path-Space Differentiable Rendering

Supplemental Materials

Kai Yan^1,2, Christoph Lassner², Brian Budge², Zhao Dong², and Shuang Zhao^{1 1}University of California, Irvine          ²Meta Reality Labs

We compare (boundary-component-only) gradients estimated with our method and PSDR [Zhang et al. 2020] at equal time and storage. The references are generated using PSDR with very high sample counts.

The derivatives and absolute differences are visualized using the color map below, and all visualizations in each row share the same color-map limits.

Orig image	Reference	PSDR (equal-time/storage)	Ours

We compare full gradients estimated with our method (with the interior component handled using PSDR) and warped-area sampling (WAS) [Bangaru et al. 2020] at both equal-time and equal-sample. The references are generated using PSDR with very high sample counts.

The derivatives and absolute differences are visualized using the color map below, and all visualizations in each row share the same color-map limits.

Orig image	Reference	WAS (equal-time)	WAS (equal-sample)	Ours

We conduct an ablation study to evaluate the effectiveness of our (i) multiple importance sampling, and (ii) edge sorting and remeshing processes.

	Reference	PSDR	Ours (light sampling)	Ours (dir. sampling)	Ours (MIS)
No edge sorting
With edge sorting

Scene	# Target images	# Param.	# Mini-batch size	# Iter.	Guiding memory	Guiding time (per iter.)	Rendering time (per iter.)	Postproc. time (per iter.)
Dodoco (Teaser)	70	450,000	2	3000	4 MB	1.34 s	2.59 s	0.310 s
Jumpy Dumpty	140	60,000	2	520	2 MB	0.61 s	1.11 s	0.016 s
Klee	40	60,000	1	200	2 MB	0.33 s	0.82 s	0.016 s
Kirby	35	15,000	1	400	80 KB	0.91 s	3.67 s	0.017 s
Bunny in glass	50	30,000	2	2000	128 KB	1.79 s	17.55 s	0.022 s
Bunny shadow 2	70	30,000	2	600	2 MB	0.47 s	0.72 s	0.012 s
Duck	35	3,205,728	1	400	2 MB	0.34 s	0.94 s	0.092 s
Mora	70	150,001	1	1600	4 MB	0.43 s	1.01 s	0.109 s
Glass Dodoco	140	60,000	1	700	400 KB	0.94 s	5.76 s	0.093 s
Colored Dodoco	70	150,003	2	1400	2 MB	0.76 s	1.14 s	0.182 s
Sansan	140	60,000	2	1000	2 MB	0.68 s	1.08 s	0.016 s

As a continuation of the ablation study above (1.3), we evaluate the usefulness of our multiple importance sampling and edge sorting techniques in inverse rendering. Specifically, we run identically configured inverse-rendering optimizations with gradients estimated using our primary-sample-space guiding with MIS, light sampling and direction sampling, respectively. Additionally, for each sampling method, we show results with and without edge sorting.

Jumpy Dumpty

• This scene involves a glossy object under environmental lighting.
• We solve for the shape of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).

Klee

• This scene involves a glossy object under complex environmental lighting.
• We solve for the shape of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).

We further demonstrate the usefulness of our technique by comparing inverse-rendering results with gradients estimated using our method and PSDR [Zhang et al. 2020]. Since our technique focuses on estimating the boundary component, we use identical interior components as PSDR.

For each example, both methods use identical target images (with one shown), initializations, and optimization settings (e.g., learning rates). We also configured the two methods so that they run in approximately equal-time and equal-storage.

Kirby

• This scene involves a rough-glass object lit by an area light.
• We solve for the shape of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).

Bunny in glass

• This scene involves an object inside a rough-glass cube.
• We solve for the shape of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).

Bunny shadow 2

• This scene involves an object under environmental illumination, casting a shadow on the ground.
• By looking at only shadow images of the object (with varying known orientations), we solve for the shape of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).

Teaser

• This scene contains three objects under environmental illumination.
• We jointly optimize the shapes of all three objects.

Duck

• This scene contains a textured object under environmental illumination.
• We jointly optimize the shape and the spatially varying albedo of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).

Mora

• This scene contains a glossy object under environmental illumination.
• We jointly optimize the shape of the object and the global rotation of the environment lighting (around the vertical axis).
• The mesh and rotation errors plotted below are used only for evaluation (and not for optimization).

Glass Dodoco

• This scene involves a rough-glass object under area lighting.
• We optimize the shape of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).

Colored Dodoco

• This scene involves a glossy object under environmental illumination.
• We jointly optimize the shape and the (spatially invariant) albedo of this object.
• The mesh and material errors plotted below are used only for evaluation (and not for optimization).

Sansan

• This scene involves a glossy object under environmental illumination.
• We optimize the shape of this object.
• The mesh error plotted below is used only for evaluation (and not for optimization).