Research map / Image generation / Text-to-image diffusion
Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Unify-Agent adapts the Bagel unified multimodal model into an end-to-end agent (Think, Research, Recaption, Generate) that searches text and images at inference and recaptions evidence for world-grounded image synthesis, evaluated on a new FactIP benchmark.
Problem
Text-to-image models rely on frozen parametric knowledge and fail to generate long-tail, knowledge-intensive entities (rare IPs, celebrities, landmarks) with correct identity and appearance.
Approach
Build the full agentic workflow (gap detection, sequential text-then-image search, evidence-grounded recaptioning, generation) inside a single unified multimodal model rather than a multi-API pipeline, using recaptions as a filter that turns retrieved text and reference images into identity-preserving and scene-compositional constraints.
Key results
- On FactIP, Unify-Agent reaches an overall score of 73.2 versus 50.9 for its base Bagel and 28.9 for FLUX.1-dev.
- On WiSE, Unify-Agent scores 0.77 overall, above BAGEL+CoT at 0.70 and the best open unified baseline.
- On KiTTEN, Unify-Agent achieves an overall score of 4.08 versus 3.50 for Imagen-3.
Surprising findings
- Removing the recaption stage drops FactIP overall to 62.9, and removing the VAE branch from recaptioning leaves overall at 71.2, only slightly below the full model.
- Removing the identity-preserving constraint raises the Content score on FactIP to 72.6 relative to the vanilla Bagel baseline, and the scene-compositional ablation yields a higher Relevance than the full model (68.6 vs 72.4 is lower, but Overall 68.2 vs 73.2).
Limitations
- FactIP benchmark and training trajectories are self-constructed and not released, limiting reproducibility.
- Evaluation relies on GPT-4o and Seed2.0 MLLM judges whose scoring may be biased or inconsistent.
- Training data is built using proprietary teacher models (Claude Opus 4.6, Gemini, Nano Banana Pro, GPT-4o), so the pipeline depends on closed APIs.
Experimental setup
Datasets: self-constructed IP trajectory dataset (143K trajectory-image pairs) WiSE KiTTEN T2I-FactualBench
Benchmarks: FactIP WiSE KiTTEN T2I-FactualBench
Metrics: FactIP weighted overall score (Clarity, Content, Aesthetics, WiScore KiTTEN text and entity alignment T2I-FactBench concept and composition factuality
Baselines: GPT-Image-1.5 Seedream-5 Nano Banana-2 FLUX.1-dev SD-3.5-large Qwen-Image Bagel Bagel-CoT Emu3.5 Hunyuan-Image-3.0 Echo4o Imagen-3
Builds on: Bagel Chain-of-thought rectified flow reject sampling
Closest papers by meaning
- AgentVista: Evaluating Multimodal Agent in Ultra-Challenging Realistic Visual Scenarios (NeurIPS 2026 · similarity 0.89)
- VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents (ICLR 2025 · similarity 0.88)
- Gen-Searcher: Reinforcing Agentic Search for Image Generation (NeurIPS 2026 · similarity 0.87)
- DeepEyesV2: Toward Agentic Multimodal Model (ICLR 2026 · similarity 0.87)
- ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.87)
- Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.