Research map / Image generation / Fast diffusion sampling
Visual Generation Without Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Guidance-Free Training (GFT), which trains a single visual generative model with a CFG-style maximum likelihood loss so that sampling needs no unconditional model, matching CFG quality at half the sampling cost across diffusion, AR and masked models.
Problem
Classifier-free guidance doubles sampling cost because it needs both conditional and unconditional model passes, and it complicates post-training (distillation, RLHF) of visual generative models.
Approach
Parameterize the conditional model implicitly as a pseudo-temperature-weighted interpolation of a guidance-free sampling network and an unconditional network (beta*eps_s + (1-beta)*sg[eps_u]), so the same conditional maximum-likelihood loss directly trains an explicit guidance-free sampler conditioned on beta, usable from scratch or by fine-tuning.
Key results
- DiT-XL/2 fine-tuned with GFT reaches guidance-free FID 1.99 on ImageNet 256x256 vs CFG 2.11 with only 2% of pretraining epochs.
- From-scratch GFT on LlamaGen-L gives FID 2.52 without guidance vs CFG 3.06, and on DiT-B/2 gives 9.04 vs 9.72.
- GFT halves sampling cost since only one model is evaluated per step, with about 19% extra training time per update versus CFG.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune DiT-XL/2 with GFT instead of guidance distillation | DiT-XL/2 · ImageNet-256x256 | FID: 2.11 -> 1.99 | Helped |
| Fine-tune LlamaGen-3B with GFT instead of Condition Contrastive Alignment | LlamaGen-3B · ImageNet-256x256 | FID: 2.69 -> 2.21 | Helped |
| Fine-tune VAR-d30 with GFT instead of Condition Contrastive Alignment | VAR-d30 · ImageNet-256x256 | FID: 2.54 -> 1.91 | Helped |
| Fine-tune Stable Diffusion 1.5 with GFT instead of guidance distillation | Stable Diffusion 1.5 · COCO 2014 validation | FID: 8.16 -> 8.10 | Helped |
| Train DiT-B/2 from scratch with GFT instead of CFG | DiT-B/2 · ImageNet-256x256 | FID: 9.72 -> 9.04 | Helped |
| Train MAR-B from scratch with GFT instead of CFG | MAR-B · ImageNet-256x256 | IS: 260.7 -> 279.5 | Helped |
| Train LlamaGen-L from scratch with GFT instead of CFG | LlamaGen-L · ImageNet-256x256 | FID: 3.06 -> 2.52 | Helped |
| Train VAR-d16 from scratch with GFT instead of CFG | VAR-d16 · ImageNet-256x256 | FID: 3.36 -> 3.42 | Hurt |
| Use a pyramid beta schedule with alpha=1.5 rather than alpha=1.0 (ablation) | GFT VAR · ImageNet-256x256 | FID | Helped |
| Stop gradients through the unconditional-model output during training (ablation) | GFT · Training stability | training stability | Helped |
| Disable conditioning dropout during fine-tuning (ablation) | Stable Diffusion 1.5 GFT · COCO 2014 validation | CLIP score | Helped |
| Vary the GFT sampling temperature beta (ablation) | GFT · ImageNet and COCO-2014 validation | FID-IS / FID-CLIP trade-off | Mixed |
Surprising findings
- Guidance-free GFT models trained from scratch outperform their fine-tuned counterparts on DiT-B/2, MAR-B, and LlamaGen-L.
- Vanilla temperature scaling of logits or outputs is generally ineffective for visual sampling, even for AR models.
Limitations
- Large-scale from-scratch experiments are limited because most from-scratch experiments use smaller models (e.g., DiT-B/2, MAR-B, LlamaGen-L, VAR-d16) due to computational constraints.
- Inference beta schedules for MAR and VAR were tuned by hand to maximize performance, and the paper notes beta may not precisely capture the effect of guidance scale after training.
Experimental setup
Datasets: ImageNet-256x256 LAION-Aesthetic 5+ (18M subset) COCO 2014
Benchmarks: ImageNet-256x256 COCO 2014 validation (zero-shot)
Metrics: FID IS CLIP score precision-recall
Baselines: Classifier-free guidance (CFG) Guidance distillation Condition Contrastive Alignment (CCA) DiT-XL/2 VAR-d30 LlamaGen-3B MAR-B Stable Diffusion 1.5
Builds on: classifier-free guidance guidance distillation DiT VAR LlamaGen
Closest papers by meaning
- Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment (ICLR 2025 · similarity 0.91)
- Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models (ICLR 2025 · similarity 0.89)
- Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking (NeurIPS 2025 · similarity 0.88)
- Understanding and Improving Training-free Loss-based Diffusion Guidance (NeurIPS 2024 · similarity 0.88)
- MaskSense: Confronting the Visual Exploration Trap in Masked Image Generation (NeurIPS 2026 · similarity 0.88)
- Feedback Guidance of Diffusion Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.