Research map / Vision-language and multimodal / Multimodal LLMs

Factuality Matters: When Image Generation and Editing Meet Structured Visuals

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds a 1.3M-pair code-aligned structured-visual editing/generation dataset with CoT annotations, trains a FLUX.1 Kontext + Qwen-VL MLP-connector unified model via three-stage curriculum, and introduces StructBench with the StructScore Q&A metric.

Problem

Modern visual generation and editing models struggle with structured visuals (charts, diagrams, math figures) that require composition planning, precise text rendering, and multimodal reasoning for factual fidelity, and there is no suitable training data or fine-grained evaluation for them.

Approach

Derive code-aligned image editing pairs by editing executable drawing programs and rendering them, add GPT-5 CoT annotations, connect Qwen-VL features to FLUX.1 Kontext via a lightweight MLP connector trained in three stages, and score outputs with an atomic multi-round Q&A metric (StructScore) that compares VLM open-ended answers to ground truth.

Key results

• On StructEditBench, the proposed model reaches 55.98% overall accuracy, the best among open-source models and second only to Nano Banana 2.0 (67.05%) in the reported table.
• On StructT2IBench, the proposed model scores 28.80% overall, versus 41.03% for Qwen-Image and 49.58% for GPT-Image.
• StructScore correlates with human Elo at r>0.9 with GPT-5 as evaluator, versus much lower correlation for PSNR.

Surprising findings

• PSNR and SSIM correlate poorly with human preference on structured visuals, while semantic Q&A accuracy correlates strongly.
• Adding an external GPT-5 reasoner at inference lets Bagel (38.44) outperform its native thinking variant Bagel-Think (33.34).

Limitations

• Even the best model reaches only about half accuracy on generation or editing.
• Qwen2.5-VL-72B judge aligns less well with human preference than GPT-5.
• Inference-time reasoning relies on a closed-source GPT-5 reasoner.

Experimental setup

Datasets: code-aligned structured image editing pairs (1.3M) ImgEdit-style general editing data open-source text-to-image datasets

Benchmarks: StructBench (StructEditBench) StructT2IBench ImgEdit

Metrics: StructScore accuracy PSNR SSIM human Elo

Baselines: Nano Banana GPT-Image Seedream 4.0 Bagel Bagel-Think Qwen-Image FLUX.1 Kontext Step1X-Edit OmniGen2

Builds on: FLUX.1 Kontext MetaQuery InstructPix2Pix

Closest papers by meaning

• GIR-Bench: Versatile Benchmark for Generating Images with Reasoning (ICLR 2026 · similarity 0.89)
• MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning (NeurIPS 2025 · similarity 0.89)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.89)
• Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing (NeurIPS 2025 · similarity 0.89)
• Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought (ICML 2026 · similarity 0.88)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.88)