Research map / Image generation / Autoregressive and latent generation

AR-RAG: Autoregressive Retrieval Augmentation for Image Generation

NeurIPS 2025 · Image generation › Autoregressive and latent generation · approach: Retrieval and matching with generative features · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AR-RAG, which autoregressively retrieves patch-level k-NN references during image generation, instantiated as training-free DDM decoding and fine-tuned SFB modules on Janus-Pro and Show-o.

Problem

Static image-level retrieval for image generation injects fixed, often irrelevant reference images that cause over-copying, stylistic bias, and poor compositional instruction following.

Approach

Performs context-aware k-NN retrieval at the patch level at every autoregressive step, using already-generated surrounding patches as queries against a database of patch embeddings keyed by their h-hop neighborhoods; DDM merges retrieved-token distributions training-free, while SFB smooths and blends retrieved features via multi-scale convolutions and compatibility-weighted fusion.

Key results

• Janus-Pro + SFB reaches 0.78 GenEval overall vs 0.71 for Janus-Pro and 0.65 for RA-CM3-style baseline.
• Janus-Pro + SFB achieves FID 6.67 on Midjourney-30K vs 14.33 for Janus-Pro.
• On DPG-Bench, Janus-Pro + SFB scores 79.36 overall vs 77.26 for Janus-Pro; DDM gives 77.88 with no training.

Measured effects reported in the paper

Surprising findings

• Prior image-level retrieval methods (RDM, ImageRAG) underperform their non-retrieval counterparts on GenEval and DPG-Bench.
• The training-free DDM adds only 0.22% inference overhead, while SFB adds 36.03%.

Limitations

• Relies on discrete image tokenization, so not directly applicable to continuous latent diffusion models.
• Retrieval database (about 13.6M images) is far smaller than billion-scale databases, which may introduce visual pattern biases.
• Only 2D image generation was explored.

Experimental setup

Datasets: CC12M JourneyDB DataComp Midjourney-v6

Benchmarks: GenEval DPG-Bench Midjourney-30K

Metrics: GenEval overall score DPG-Bench overall score FID CMMD FWD inference time

Baselines: Janus-Pro Show-o RA-CM3 ImageRAG RDM LlamaGen LDM Stable Diffusion PixArt-alpha DALL-E 2 SDXL SD3

Builds on: Janus-Pro Show-o RA-CM3 ImageRAG FAISS

Closest papers by meaning

• Retrieval-Augmented Perception: High-resolution Image Perception Meets Visual RAG (ICML 2025 · similarity 0.88)
• Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation (NeurIPS 2025 · similarity 0.88)
• Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching (ICLR 2025 · similarity 0.87)
• LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding (ICLR 2025 · similarity 0.87)
• RealRAG: Retrieval-augmented Realistic Image Generation via Self-reflective Contrastive Learning (ICML 2025 · similarity 0.87)
• RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering (NeurIPS 2025 · similarity 0.86)