Research map / Image generation / Text-to-image diffusion
Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that consistency-distilled text-to-image students sometimes beat their SD1.5/SDXL teachers, and proposes an adaptive pipeline where an ImageReward-based oracle decides whether a student sample is refined or regenerated by the teacher.
Problem
Few-step distilled text-to-image diffusion models generally fall short of their teachers in quality, limiting practical use; the paper asks whether students can be exploited rather than replaced.
Approach
Student samples are often better than the teacher's, especially where they diverge from the teacher. An adaptive collaboration uses the student first, accepts samples whose ImageReward score exceeds a percentile threshold, and otherwise improves them with the teacher via SDEdit-style refinement or regeneration.
Key results
- Adaptive pipeline reaches SD1.5 teacher-level human preference with up to 5x fewer steps than DDIM-50 and up to 2.5x fewer than DPM-25.
- Adaptive method improves generative performance by up to 19% and 20% over the teacher and the refinement-only baseline at equal average steps.
- With ADD-XL, the adaptive scheme improves human preference by 14%; with CD-SDXL it matches SDXL quality while being about 5x more efficient on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive student selection and teacher refinement instead of refining every student sample (ablation) | SD1.5 · Text-to-image generation, same average inference steps | Human preference: up to 20% | Helped |
| Use adaptive student-teacher collaboration | SD1.5 · Text-to-image generation, same average inference steps | Human preference: up to 19% | Helped |
| Use adaptive refinement with CD-SDXL | SDXL · Text-to-image generation | Human preference: 5× | Helped |
| Use adaptive refinement with ADD-XL | ADD-XL · Text-to-image generation | Human preference: 14% | Helped |
| Use adaptive regeneration for controllable generation with Canny edges | ControlNet with CD-SD1.5 · Canny-edge conditioned image generation | Human preference: 19% | Helped |
| Use adaptive refinement for controllable generation with segmentation masks | ControlNet with CD-SD1.5 · Semantic-segmentation-conditioned image generation | Human preference: 4% | Helped |
| Use adaptive regeneration for text-guided image editing | SDEdit with SD1.5 · Text-guided image editing, strength 0.6 | ImageReward: 0.179 ± .020 -> 0.281 ± .008 | Helped |
| Use adaptive regeneration rather than adaptive refinement for image editing (ablation) | SDEdit with SD1.5 · Text-guided image editing | DINOv2 reference preservation | Helped |
| Use adaptive regeneration rather than adaptive refinement (ablation) | SD1.5 text-to-image generation · COCO2014 text prompts | CLIP score | Helped |
| Use adaptive refinement rather than adaptive regeneration (ablation) | SD1.5 text-to-image generation · COCO2014 text prompts | FID | Helped |
| Use the student sample when it is preferred over the teacher | Consistency-distilled SD1.5 · COCO2014 text-to-image samples | Human preference: ∼30% | Mixed |
| Use adaptive regeneration instead of the teacher alone | SD1.5 · SDEdit, strength 0.6 | DINOv2 reference preservation: 0.667 ± .005 -> 0.669 ± .006 | No clear effect |
Surprising findings
- About 30% of distilled student samples were preferred over the teacher's samples in human votes on SD1.5.
- Student wins concentrate where the student diverges most from the teacher, so student-teacher distance is informative about student superiority.
Limitations
- Relies on an automatic quality estimator (ImageReward) as the oracle, whose accuracy limits gains.
- Gains depend on tuning rollback sigma and threshold tau per inference budget on a hold-out set.
- Human evaluation uses professional assessors on a limited set of 600 prompts per setting.
Experimental setup
Datasets: LAION-2B 80M subset COCO2014 LAION-Aesthetics
Benchmarks: COCO2014 5K validation LAION-Aesthetics prompts
Metrics: FID CLIP score ImageReward human preference win rate DINOv2 distance
Baselines: SD1.5 with DDIM-50 SD1.5 with DPM-Solver-25 CD-SD1.5 Refinement without adaptive step Restart Sampling SDXL with DDIM-50 ADD-XL CD-SDXL
Builds on: consistency distillation ImageReward SDEdit DPM-Solver ControlNet
Closest papers by meaning
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.90)
- SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation (CVPR 2024 · similarity 0.90)
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.89)
- Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization (ICLR 2025 · similarity 0.89)
- Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generation (ICLR 2025 · similarity 0.88)
- OneActor: Consistent Subject Generation via Cluster-Conditioned Guidance (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.