Research map / Probabilistic and scientific modeling / Variational and generative sampling

Variance Reduction for Expectations with Diffusion Teachers

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Importance sampling and quadrature · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CARV, a compute-aware variance-reduction framework for Monte Carlo gradients from frozen diffusion teachers, combining amortized noise resampling, timestep importance sampling and stratified-inverse-CDF sampling, yielding 2-3x effective compute gains in text-to-3D SDS and video data attribution but no FID gain in DMD distillation.

Problem

Monte Carlo gradient estimators from frozen diffusion teachers (SDS for text-to-3D, DMD for one-step distillation, gradient-based data attribution) have high variance, and each draw is expensive due to upstream rendering, simulation or encoding, so compute cost is dominated by estimator variance.

Approach

Cache expensive upstream computation (render/encode/generate) and amortize it over many cheap diffusion-noise resamples, combined with timestep importance sampling using the SDS weight and stratified inverse-CDF sampling; all are unbiased drop-ins evaluated by compute-aware efficiency metrics (ECM, RE).

Key results

• Reuse alone yields about 2.6x effective compute multiplier in text-to-3D SDS, and IW+Strat about 3.3x (ECM peak at K=8).
• In DMD, (8,16) resampling cuts parameter-gradient variance from 982 to 59.9 (IID), and stratification to 30.7, but FID does not improve at matched wall-clock.
• For video data attribution, stratified sampling achieves more than 2x effective compute multipliers at reasonable budgets (e.g., 2.85x at 256 timesteps).

Surprising findings

• Variance reduction in single-step DMD distillation (order-of-magnitude gradient variance cut) does not improve downstream FID at matched wall-clock, indicating MC variance is not the bottleneck there.
• Importance sampling via SDS weight gives only about 1.2x variance reduction, smaller than amortized reuse gains.

Limitations

• In DMD, variance reduction does not translate to FID gains; auxiliary losses and generator input diversity dominate convergence.
• Reported gains are most pronounced when render cost exceeds denoise cost; benefit is smaller for attribution where encoding is moderate.
• ECM relies on interpolation/extrapolation along the baseline Pareto curve using the Monte Carlo variance rate.

Experimental setup

Datasets: ImageNet-256 VIDGEN-1M

Benchmarks: FID (DMD ImageNet-256) CLIP score (text-to-3D) ranking correlation (video attribution)

Metrics: effective compute multiplier relative efficiency gradient variance FID CLIP score Pearson correlation with ground-truth gradients

Baselines: uniform IID SDS (DreamFusion/Magic3D/ProlificDreamer config) DMD IID (8,1) uniform IID attribution sampling

Builds on: Score Distillation Sampling Distribution Matching Distillation MOTIVE Welford's online algorithm Variational Score Distillation

Closest papers by meaning

• Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance (NeurIPS 2026 · similarity 0.89)
• Elucidating the Exposure Bias in Diffusion Models (ICLR 2024 · similarity 0.89)
• EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.88)
• The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.88)
• Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration (NeurIPS 2025 · similarity 0.88)
• SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.88)