Research map / Image generation / Text-to-image diffusion
You Only Sample Once: Taming One-Step Text-to-Image Synthesis by Self-Cooperative Diffusion GANs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes YOSO, a diffusion-GAN hybrid that smooths adversarial divergence via self-cooperative learning (generator's less-noisy outputs as targets), enabling one-step image and text-to-image generation fine-tuned from PixArt-α and SD 1.5.
Problem
Diffusion-GAN hybrids for one-step generation suffer from training instability, mode collapse, or inefficient one-step learning, and large-scale text-to-image one-step generation is costly to train.
Approach
Smooth the adversarial divergence by treating the generator's own one-step denoised outputs from less-corrupted inputs (t_k < t) as the target distribution (self-cooperative learning), combined with an L2 point-matching term; extend to text-to-image with latent perceptual loss, latent discriminator from pre-trained SD, informative prior initialization, and a quick adaptation stage to v-prediction/
Key results
- On CIFAR-10 unconditional, YOSO reaches FID 1.81 in one step when fine-tuning EDM, and 2.26 when trained from scratch.
- YOSO-LoRA on SD 1.5 at one step achieves HPS 28.33 and AeS 5.97, beating Hyper-SD-LoRA (28.01 HPS) without human feedback learning.
- YOSO on PixArt-alpha with one step achieves HPS 30.52 and AeS 6.19, versus DMD's 29.78 and 6.02 at one step.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add decoupled scheduler (ablation) | YOSO-base · CIFAR-10 | FID: 3.82 to 3.30 | Helped |
| add decoupled scheduler (ablation) | YOSO-base · Text-to-image | HPS: 26.42 to 27.10 | Helped |
| remove annealing strategy (ablation) | YOSO · CIFAR-10 | FID: 3.05 to 3.30 | Hurt |
| remove annealing strategy (ablation) | YOSO · Text-to-image | HPS: 28.33 to 27.10 | Hurt |
| replace cooperative adversarial loss with real-data adversarial divergence (ablation) | YOSO-base · CIFAR-10 | FID: 3.82 to 10.85 | Hurt |
| replace cooperative adversarial loss with real-data adversarial divergence (ablation) | YOSO-base · Text-to-image | HPS: 26.42 to 24.88 | Hurt |
| replace YOSO cooperative adversarial loss with corrupted-data adversarial divergence (ablation) | YOSO · CIFAR-10 | FID: 3.82 to 45.15 | Hurt |
| use latent perceptual loss for consistency loss (ablation) | text-to-image model · Text-to-image | HPS: 15.22 to 20.08 | Helped |
| use latent perceptual loss for consistency loss (ablation) | text-to-image model · Text-to-image | AeS: 5.23 to 5.45 | Helped |
| remove consistency loss (ablation) | YOSO · CIFAR-10 | FID | No clear effect |
| apply quick adaptation stage (ablation) | YOSO-PixArt-α · Text-to-image, 5k training iterations | image quality | Helped |
Surprising findings
- Fine-tuning EDM with YOSO in one step outperforms EDM itself using 36-step ODE sampling on CIFAR-10.
- Replacing the cooperative adversarial loss with standard adversarial divergence against real data causes FID to degrade from 3.82 to 10.85 and mode collapse.
Limitations
- Zero-shot 1024 resolution is obtained by a LoRA-style weight combination rather than explicit training; applying YOSO-LoRA to new base models (e.g., Dreamshaper, Toonyou) fails to produce reasonable one-step samples.
- Limited discussion of fairness, fine-grained detail, and object-count control, as acknowledged by the authors.
- Some metrics (HPS, AeS) are machine-based and may be gamed; zero-shot COCO FID is noted as unreliable.
Experimental setup
Datasets: CIFAR-10 JourneyDB ImageNet-64 FFHQ-1024 COYO-700M LAION COCO-5k
Benchmarks: CIFAR-10 ImageNet-64 FFHQ-1024 COCO-5k HPS v2.1 benchmark
Metrics: FID HPS v2.1 Aesthetic Score ImageReward CLIP score zero-shot COCO FID-30k
Baselines: SD-Turbo DMD Hyper-SD InstaFlow LCM-LoRA PeRFlow TCD-LoRA UFOGen Diffusion2GAN ADD CTM SiD
Builds on: DDGAN UFOGen consistency training EDM PixArt-alpha
Closest papers by meaning
- Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.90)
- Bridging Data Gaps in Diffusion Models with Adversarial Noise-Based Transfer Learning (ICML 2024 · similarity 0.89)
- Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding (ICML 2024 · similarity 0.89)
- SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation (CVPR 2024 · similarity 0.88)
- E$^2$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation (ICML 2024 · similarity 0.88)
- UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.