Research map / Image generation / Fast diffusion sampling

Hierarchical Variational Policies for Reward-Guided Diffusion

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Training-free diffusion acceleration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hierarchical Variational Policies (HVP) that amortize test-time reward guidance for diffusion models into a learned initial-noise predictor and per-step latent-control policy (AHVP), with a semi-amortized variant (SHVP) that adds light test-time refinement, evaluated on FFHQ/ImageNet inverse problems.

Problem

Test-time adaptation of pretrained diffusion models to downstream differentiable objectives (e.g., inverse problems) relies on expensive guidance or inner-loop optimization at inference time.

Approach

Cast guidance as a hierarchical variational policy with latent stochastic controls added to the diffusion state, and amortize it into a learned initial-noise predictor (DiT) and per-step controller (DiT) trained with a tractable ELBO, so sampling is a single controlled rollout; SHVP refines controls with a few gradient steps.

Key results

• On SR x4 FFHQ-256, AHVP reaches LPIPS 0.073 at 0.6 s versus NDTM's 0.064 at 4.1 s; SHVP achieves the best LPIPS 0.059 and FID 38.50.
• On SR x4 ImageNet-256, SHVP achieves LPIPS 0.202 and FID 65.54, the best among reported methods.
• Stage-2 per-step controls reduce AHVP LPIPS on FFHQ SR x4 from 0.147 to 0.073.

Surprising findings

• A deterministic per-step policy matches the stochastic policy on FFHQ but is substantially worse on ImageNet (LPIPS 0.328 vs 0.244).
• AHVP is worse than several baselines on PSNR/SSIM for some tasks (e.g., ImageNet SR x8 SSIM 0.371), indicating a perception-distortion trade-off.

Limitations

• Only differentiable rewards are handled; non-differentiable rewards are left to future work.
• Evaluated only on inverse problems with 100 held-out samples per dataset and unconditional FFHQ/ImageNet checkpoints.
• Amortization error requires semi-amortized refinement (SHVP) to reach best quality, increasing cost.

Experimental setup

Datasets: FFHQ ImageNet

Benchmarks: FFHQ-256 inverse problems (SR x4, SR x8, random inpainting 90%, HDR) ImageNet-256 inverse problems (SR x4, SR x8, random inpainting 90%, HDR)

Metrics: LPIPS FID PSNR SSIM runtime

Baselines: DAPS NDTM RED-Diff C-ΠGDM MPGD DDRM DPS HyperNoise

Builds on: DAPS NDTM semi-amortized variational inference DRaFT DiT

Closest papers by meaning

• Learning to Sample From Diffusion Models via Inverse Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
• Variational Control for Guidance in Diffusion Models (ICML 2025 · similarity 0.90)
• Test-time Alignment of Diffusion Models without Reward Over-optimization (ICLR 2025 · similarity 0.90)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.90)
• Inference-Time Alignment of Diffusion Models with Direct Noise Optimization (ICML 2025 · similarity 0.90)
• Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models (ICML 2024 · similarity 0.89)