Research map / Label-efficient and robust learning / Knowledge distillation
Target-Driven Distillation: Consistency Distillation with Target Timestep Selection and Decoupled Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Target-Driven Distillation (TDD), a consistency distillation method for SDXL with target timestep selection and decoupled guidance, enabling few-step generation with CFG tuning at inference.
Problem
Few-step diffusion distillation (consistency models) suffers from fixed guidance during distillation and suboptimal timestep selection, limiting quality and the ability to tune guidance at inference.
Approach
Selects target timesteps non-equidistantly (inserting 8-step timesteps into a 4-step schedule) and decouples the distillation guidance scale from inference guidance by re-deriving the noise prediction with an additional unconditional term, so CFG can be tuned to its normal range after distillation.
Key results
- Uses fixed CFG of 2.0 for TDD vs 1.0 for LCM/TCD and 1.6 for PCM in main comparison; reports image complexity comparisons with TCD and TDD, with TDD slightly lower in complexity (no specific numbers in the excerpt).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply x0 clipping at the initial step instead of using no x0 clipping (ablation) | TDD (non-adversarial), 4-step sampling · Qualitative image samples | visual realism and detail | Helped |
| apply x0 clipping at every step instead of only the first step (ablation) | TDD (non-adversarial), 4-step sampling · Qualitative image samples | image clarity | Hurt |
Surprising findings
- Applying x0 clipping at only the first step improves quality, while applying it at every step yields washed-out, blurry images.
Limitations
- Image complexity metric is not highest; authors attribute this to artifacts and instability that inflate scores.
- Recommends x0 clipping only at first step for higher CFG values, indicating sensitivity to sampling settings.
Experimental setup
Datasets: LAION-5B High-Resolution (aesthetic > 5.5 subset) COCO-2014 validation (COCO-30K, COCO-2K) PartiPrompts
Benchmarks: COCO-2014 (COCO-30K, COCO-2K) PartiPrompts
Metrics: image complexity
Baselines: LCM TCD PCM
Builds on: consistency models latent consistency models trajectory consistency distillation classifier-free guidance
Closest papers by meaning
- Distillation Scaling Laws (ICML 2025 · similarity 0.81)
- SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision (NeurIPS 2026 · similarity 0.79)
- UniADS: Universal Architecture-Distiller Search for Distillation Gap (AAAI 2024 · similarity 0.79)
- GDMD: Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.78)
- Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher (ICLR 2025 · similarity 0.78)
- Elucidating the Preconditioning in Consistency Distillation (ICLR 2025 · similarity 0.78)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.