Research map / Reinforcement learning / Policy gradient and actor-critic

ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Scheduling

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Regularized scalable deep value learning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Adaptive Reparameterized Time (ART), a control formulation of diffusion timestep scheduling, solved via a continuous-time RL actor-critic (ART-RL) with a two-directional bridge between the deterministic optimum and Gaussian policies; the distilled schedule improves FID on CIFAR-10 EDM and transfers to other datasets.

Problem

Choosing timestep grids for diffusion sampling under a fixed step budget: uniform or hand-crafted grids can be suboptimal and minimize Euler discretization error poorly.

Approach

Treats the time-warping rate of the sampling clock as a control that redistributes steps to minimize an aggregate Euler local-error surrogate; randomizes it as Gaussian policies to apply continuous-time actor-critic RL, with theorems showing the deterministic ART optimum equals the mean of optimal Gaussian policies.</non>

Key results

• On CIFAR-10 with Heun sampler at NFE=35, ART-RL reaches FID 1.82 vs 1.85 for EDM and 1.89 for DPM-logSNR.
• At NFE=3 on CIFAR-10, ART-RL FID 152.86 vs 465.83 for EDM/DPM-logSNR/AYS.
• Cross-dataset transfer of the CIFAR-10 schedule improves FID at low NFE, e.g. ImageNet NFE=9: 29.49 vs 35.32 for EDM.

Surprising findings

• The CIFAR-10 schedule learned once transfers without retraining to AFHQv2, FFHQ, and ImageNet with gains at low NFE.

Limitations

• Theory relies on Euler local-error surrogate; training uses Euler while evaluation uses Heun.
• Gains at the largest budget are marginal (e.g. 1.82 vs 1.85 at NFE=35).
• Restricted to deterministic time-only grids distilled from the stochastic policy; state-dependent schedules not explored.

Experimental setup

Datasets: CIFAR-10 AFHQv2 FFHQ ImageNet

Benchmarks: CIFAR-10 AFHQv2 FFHQ ImageNet 1D analytic Gaussian score

Metrics: FID W2

Baselines: Uniform DPM-logSNR EDM AYS

Builds on: EDM DPM-Solver continuous-time reinforcement learning (exploratory HJB) AYS

Closest papers by meaning

• Learning to Sample From Diffusion Models via Inverse Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
• Prior-Guided Diffusion Planning for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• Adaptive Non-Uniform Timestep Sampling for Accelerating Diffusion Model Training (CVPR 2025 · similarity 0.88)
• Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View (NeurIPS 2026 · similarity 0.88)
• FASTER: Value-Guided Sampling for Fast RL (NeurIPS 2026 · similarity 0.88)
• Hierarchical Variational Policies for Reward-Guided Diffusion (NeurIPS 2026 · similarity 0.88)