Research map / Image generation / Text-to-image diffusion

TerraVis: Towards Evaluation of World-Grounded Visual Consistency in Text-to-Image Generation via MLLM Workflows

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: T2I evaluation and controllable adapters · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TerraVis, a taxonomy-guided multi-stage MLLM workflow (eligibility, 18-type violation detection, major/minor severity, exponential aggregation) that scores world-grounded visual consistency of text-to-image outputs and correlates better with human judgments than existing metrics.

Problem

Existing text-to-image metrics (quality, aesthetics, alignment, preference) do not capture real-world implausibilities such as malformed anatomy, impossible object structure, physically implausible interactions, and impossible spatial relations in visually appealing images.

Approach

Define world-grounded visual consistency with an 18-type, three-level (object, interaction, scene) violation taxonomy and score images by decomposing evaluation into eligibility checking, atomic violation QA, severity classification, and a severity-weighted exponential aggregation computed deterministically.

Key results

• TerraVis achieves pooled Spearman rho=0.44 and Kendall tau=0.37 on COCO-T2I, versus 0.29 for the best existing metric (RAHF) and 0.28 for TerraVis-H.
• On GenAI-Bench, TerraVis reaches pooled rho=0.43, compared with 0.37 for RAHF.
• Gemma 4 31B as an open-source evaluator achieves rho=0.42 on COCO-T2I and 0.51 on GenAI-Bench, comparable to GPT-5.5.

Surprising findings

• GPTScore, prompting an MLLM directly for a 1-5 world-consistency rating, shows near-zero correlation with humans, assigning almost uniformly high scores.
• A holistic MLLM prompt with the full annotator instructions (TerraVis-H) detects only 8.7% of COCO-T2I images as violating, versus a 48.6% human positive rate, indicating low recall.

Limitations

• Multi-stage workflow is less efficient than single-pass scoring metrics.
• Violation detection false positives and missed violations in dense scenes remain a bottleneck.
• Taxonomy may not cover all world-consistency violations; human annotation ambiguity limits reference quality.

Experimental setup

Datasets: COCO-T2I GenAI-Bench MSCOCO

Benchmarks: COCO-T2I GenAI-Bench

Metrics: Spearman's rho Kendall's tau Pearson's r precision recall

Baselines: MUSIQ LAION-Aesthetic CLIPScore TIFA VQAScore ImageReward RAHF HPSv3 GPTScore TerraVis-H

Builds on: TIFA VQAScore GPTScore CLIPScore

Closest papers by meaning

• LoopNav: Benchmarking Spatial Consistency in World Models (NeurIPS 2026 · similarity 0.86)
• MMLongCite: A Benchmark for Evaluating Faithfulness of Long-Context Vision-Language Models (NeurIPS 2026 · similarity 0.85)
• Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation (NeurIPS 2025 · similarity 0.85)
• Evaluating Multiview Object Consistency in Humans and Image Models (NeurIPS 2024 · similarity 0.85)
• Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2) (NeurIPS 2024 · similarity 0.85)
• PoSh: Using Scene Graphs to Guide LLMs-as-a-Judge for Detailed Image Descriptions (ICLR 2026 · similarity 0.85)