Research map / Image generation / Fast diffusion sampling
Distilled Decoding 2: One-step Sampling of Image Auto-regressive Models with Conditional Score Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Distilled Decoding 2 (DD2), which distills a multi-step image AR model (VAR, LlamaGen) into a one-step generator by matching per-token conditional scores via a conditional score distillation loss, without a predefined noise-to-data mapping.
Problem
Image autoregressive models need many sequential sampling steps, making generation slow; the prior one-step method DD1 has a large quality drop and relies on a predefined noise-to-data mapping.
Approach
Treat the AR teacher's next-token probability vector as a conditional score in the codebook embedding space (via flow matching with RectFlow noise schedule), and apply SiD-style score distillation at every token position conditioned on previous clean tokens, with a separate guidance network learning the generator's conditional score; no predefined mapping is needed.
Key results
- One-step DD2 reaches FID 5.43 on VAR-d20 (teacher 3.40) and 8.59 on LlamaGen-L (teacher 4.11, 238x speedup), vs DD1 one-step FIDs of 9.55 and 11.35.
- DD2 reduces the gap between one-step and teacher by up to 67% compared with DD1 on VAR models.
- DD2 achieves up to 12.3x training speed-up over DD1 (LlamaGen-L: 52.6 vs 647.7 8xA800 GPU hours).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill VAR-d16 to 1 step with DD2 instead of DD1 | VAR-d16 · ImageNet-256 | FID: 9.94 -> 6.21 | Helped |
| Distill VAR-d20 to 1 step with DD2 instead of DD1 | VAR-d20 · ImageNet-256 | FID: 9.55 -> 5.43 | Helped |
| Distill VAR-d24 to 1 step with DD2 instead of DD1 | VAR-d24 · ImageNet-256 | FID: 8.92 -> 5.06 | Helped |
| Distill LlamaGen-L to 1 step with DD2 instead of DD1 | LlamaGen-L · ImageNet-256 | FID: 11.35 -> 8.59 | Helped |
| Refine the last tokens with the pretrained VAR model using more sampling steps | DD2 VAR-d16 · ImageNet-256 | FID: 5.24 -> 4.88 -> 4.47 | Helped |
| Add tuned AR-diffusion initialization to the generator (ablation) | DD2 LlamaGen-L · ImageNet-256, FID-5k | FID: 16.08 -> 14.77 | Helped |
| Add tuned AR-diffusion initialization to the guidance network (ablation) | DD2 LlamaGen-L · ImageNet-256, FID-5k | FID: 21.76 -> 14.77 | Helped |
| Add tuned AR-diffusion initialization to the generator (ablation) | DD2 VAR-d24 · ImageNet-256, FID-5k | FID: Collapse(>200) -> 11.53 | Helped |
| Add tuned AR-diffusion initialization to the guidance network (ablation) | DD2 VAR-d24 · ImageNet-256, FID-5k | FID: Collapse(>200) -> 11.53 | Helped |
| Replace traditional diffusion loss with GTS loss (ablation) | AR-diffusion VAR-d16 initialization model · ImageNet-256, FID-5k | FID: 17.98 -> 11.41 | Helped |
| Add performance alignment (ablation) | DD2 VAR-d16 · ImageNet-256 | FID: 8.35 -> 6.21 | Helped |
| Add performance alignment (ablation) | DD2 VAR-d20 · ImageNet-256 | FID: 6.57 -> 5.43 | Helped |
Surprising findings
- Initializing only one of generator or guidance network from the tuned AR-diffusion model causes severe degradation or collapse on VAR-d24 (collapse above FID 200).
- Set-prediction one-step baselines fail completely (FID 157.5 for VAR, 220.2 for LlamaGen), showing token dependencies matter.
Limitations
- Remaining performance gap to the teacher AR model (FID rises from 3.40 to 5.43 on VAR-d20 and 4.11 to 8.59 on LlamaGen-L in one step).
- Only evaluated on class-conditional ImageNet-256 with VQ-based VAR and LlamaGen; not extended to continuous-space or text-to-image AR models.
- Results depend on initialization; missing proper initialization of either network leads to degradation or collapse.
Experimental setup
Datasets: ImageNet-256
Benchmarks: ImageNet-256
Metrics: FID IS Precision Recall wall-clock time training GPU hours PPL
Baselines: Distilled Decoding 1 VAR skip-last steps set prediction one-step StyleGAN-XL MaskGIT DiT-L/2 LDM-4-G ADM Shortcut model Consistency distillation Progressive distillation Reflow
Builds on: Distilled Decoding 1 Diffusion score distillation (DMD) SiD MAR DMD2
Closest papers by meaning
- Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching (ICLR 2025 · similarity 0.95)
- LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding (ICLR 2025 · similarity 0.88)
- Improved Distribution Matching Distillation for Fast Image Synthesis (NeurIPS 2024 · similarity 0.88)
- Autoregressive Distillation of Diffusion Transformers (CVPR 2025 · similarity 0.87)
- When Worse is Better: Navigating the Compression Generation Trade-off In Visual Tokenization (NeurIPS 2025 · similarity 0.87)
- Unlocking Dataset Distillation with Diffusion Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.