Research map / Image generation / Text-to-image diffusion

UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs

CVPR 2024 · Image generation › Text-to-image diffusion · approach: Compositional text-to-image guidance · 59 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UFOGen, a diffusion-GAN hybrid initialized from Stable Diffusion 1.5 that generates text-conditioned images in a single step, with applications to image-to-image and controllable generation.

Problem

Text-to-image diffusion models need many sampling steps, and prior few-step methods (ODE solvers, distillation) struggle to reach one-step generation at large scale.

Approach

Combines diffusion and GAN objectives: the generator predicts x0 and samples x_{t-1} from the forward process, with an explicit x0 reconstruction loss so one-step sampling works; the generator and discriminator are both initialized from a pre-trained SD UNet.

Key results

• One-step UFOGen reaches FID-5k 22.5 and CLIP 0.311 on MS-COCO 2017, beating one-step InstaFlow-0.9B (23.4, 0.304) and two-step progressive distillation (26.0).
• On MS-COCO 2014 30k, UFOGen obtains FID 12.78 at 0.09 s, versus InstaFlow-0.9B at 13.10 and GigaGAN at 9.09 (0.13 s, 1B params).
• Ablation: with the SIDDM objective, one-step FID degrades to 28.0 while UFOGen's objective gives 22.5 in one step.

Measured effects reported in the paper

Surprising findings

• The authors argue MS-COCO zero-shot FID may be unreliable, so they rely on qualitative comparisons where UFOGen clearly beats InstaFlow.
• Standard SIDDM diffusion-GAN training fails at one-step sampling even on 2-D toy data, requiring the reconstruction-term change.

Limitations

• Zoo-level FID comparisons are not directly comparable across architectures and training data, as acknowledged by the authors.
• Discriminator adds extra parameters and training cost; training details such as compute and iteration count are not reported in the main text.

Experimental setup

Datasets: LAION-Aesthetics-6+ (LAION-5B subset) MS-COCO 2017 5k (evaluation) MS-COCO 2014 30k (evaluation)

Benchmarks: MS-COCO 2017 5k MS-COCO 2014 30k

Metrics: FID CLIP score inference time

Baselines: Progressive Distillation CFG-Aware Distillation InstaFlow-0.9B InstaFlow-1.7B Latent Consistency Model SIDDM DPM Solver

Builds on: DDGAN SIDDM Stable Diffusion SDEdit ControlNet

Closest papers by meaning

• You Only Sample Once: Taming One-Step Text-to-Image Synthesis by Self-Cooperative Diffusion GANs (ICLR 2025 · similarity 0.88)
• Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.87)
• Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis (CVPR 2025 · similarity 0.87)
• Minority-Focused Text-to-Image Generation via Prompt Optimization (CVPR 2025 · similarity 0.87)
• R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation (ICLR 2024 · similarity 0.87)
• IT3D: Improved Text-to-3D Generation with Explicit View Synthesis (AAAI 2024 · similarity 0.87)