Research map / Vision-language and multimodal / Multimodal LLMs

Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment

ICLR 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Condition Contrastive Alignment (CCA), a one-epoch fine-tuning method on pretraining data that makes autoregressive visual generators match classifier-free-guidance sample quality without guided sampling.

Problem

Autoregressive visual generation relies on classifier-free guidance, which doubles sampling cost and creates a training/design inconsistency with language models that do not use guidance.

Approach

Derives a noise-contrastive objective whose optimum equals the CFG-style guided target distribution, parameterizing the conditional residual by the difference between a fine-tuned model and the frozen pretrained model, using mismatched image-condition pairs from the pretraining data as negatives. This lets a single model be fine-tuned to sample the guided distribution directly with no change to sa

Key results

• LlamaGen-L guidance-free FID improves from 19.00 to 3.43 and IS from 64.7 to 288.2 with CCA, comparable to CFG.
• VAR-d24 guidance-free FID improves from 6.20 to 2.63 with IS 298.8 after CCA fine-tuning.
• DPO applied naively collapses LlamaGen-L (FID 61.69), while CCA outperforms Unlearning and DPO baselines.

Measured effects reported in the paper

Surprising findings

• Naive DPO on image-condition pairs degrades performance and collapses LlamaGen-L, as positive-data likelihood keeps decreasing.
• Adjusting the training parameter lambda gives FID-IS trade-offs similar to CFG scale, while adjusting beta is less predictable.

Limitations

• Guidance-free CCA results still lag behind CFG-enhanced performance on the reported metrics.
• Evaluated only on class-conditional ImageNet; text-conditioned generation is not tested.
• Adjusting beta is less effective and less predictable than adjusting lambda.

Experimental setup

Datasets: ImageNet

Benchmarks: ImageNet 256x256 class-conditional generation

Metrics: FID IS Precision Recall sFID

Baselines: CFG DPO Unlearning ADM LDM-4 U-ViT-H/2 DiT-XL/2 MDTv2-XL/2 MaskGIT MAGVIT-v2 MAGE VQGAN

Builds on: classifier-free guidance direct preference optimization noise contrastive estimation LlamaGen VAR

Closest papers by meaning

• Visual Generation Without Guidance (ICML 2025 · similarity 0.91)
• OmniGen-AR: AutoRegressive Any-to-Image Generation (NeurIPS 2025 · similarity 0.87)
• Guiding Visual Autoregressive Models through Spectrum Weakening (NeurIPS 2026 · similarity 0.87)
• EditAR: Unified Conditional Generation with Autoregressive Models (CVPR 2025 · similarity 0.87)
• Training-free Dense-Aligned Diffusion Guidance for Modular Conditional Image Synthesis (CVPR 2025 · similarity 0.87)
• ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer (ICLR 2025 · similarity 0.87)