Research map / Vision-language and multimodal / Multimodal LLMs
UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UFO, an atomized chain-of-evaluation metric that decomposes omni-condition (text+image) alignment into weighted atomic units checked via GPT-4o VQA or ArcFace, plus UFO-Bench for subject-driven customization.
Problem
Existing evaluators for subject-driven customization (embedding-based like CLIP/DINO and MLLM-based like VIEScore/DreamBench++) assess textual and visual alignment in isolation, which misaligns with human judgments when edits intentionally change subject attributes.
Approach
Decompose the evaluation into atomic evaluation units (AEUs) at local and global levels, classify each AEU by modality relevance (image-only, text-only, or both), score it via VLM yes/no VQA or dedicated function calls (ArcFace for faces), and aggregate with VLM-assigned importance weights.
Key results
- UFO reaches Spearman correlation 0.6889 with human rankings versus 0.5977 for the best baseline (DreamBench++), reported as a 15.26% improvement.
- On UFO-Bench, Doubao (2025-12) achieves the top total score of 0.7439, with Qwen-Image second at 0.7252.
- Ablation: removing AEU decomposition drops correlation from 0.6889 to 0.5752; removing weighted aggregation drops it to 0.6253.
Surprising findings
- CLIP-T shows a negative correlation with human preference (-0.1996) in subject-driven evaluation.
- Qwen-Image surpasses the closed-source Nano-Banana on the UFO-Bench total score despite a weak local-editing score.
Limitations
- Human evaluation covers only 100 test cases and six models.
- Relies on proprietary GPT-4o as judge, with the evaluation limited to personalized image generation settings.
- Benchmark is small (86 reference images), and the paper's text reports inconsistent case counts (660 cases vs. 2,640 instances).
Experimental setup
Datasets: UFO-Bench
Benchmarks: UFO-Bench
Metrics: Spearman rank correlation with human rankings UFO score
Baselines: CLIP-I CLIP-T DINO VIEScore DreamBench++
Builds on: VIEScore DreamBench++ ArcFace GPT-4o
Closest papers by meaning
- MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.87)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.86)
- Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment (ICLR 2025 · similarity 0.85)
- Training-free Dense-Aligned Diffusion Guidance for Modular Conditional Image Synthesis (CVPR 2025 · similarity 0.85)
- GMAIL: Generative Modality Alignment for generated Image Learning (ICML 2025 · similarity 0.84)
- ImagenHub: Standardizing the evaluation of conditional image generation models (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.