Research map / Vision-language and multimodal / Multimodal LLMs

VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VCG-Bench, a multi-domain benchmark (1,449 diagrams) for generating (image-to-mxGraph XML) and instruction-based editing of Draw.io diagrams via a Diagram-as-Code paradigm, evaluating closed and open VLMs with ESR, SCS, CodeXQA, and XDRFR metrics.

Problem

VLMs struggle with structured, controllable diagram generation and editing; pixel-based synthesis is hard to edit and yields distortions and hallucinated elements, and no unified multi-domain benchmark evaluates executable diagram code generation and editing.

Approach

Treat diagrams as symbolic mxGraph XML ("Diagram-as-Code"), unify image-to-XML generation (Task 1) and JSON-patch-based instruction editing (Task 2), and evaluate with executability, style consistency judged by a VLM, XML-based QA, and decomposed requirement verification on XML.

Key results

• Gemini-3-Pro achieves the best Task 1 SCS (0.7790), CodeXQA (0.9313) and ESR (0.9626) among evaluated models; GLM-4.6V leads open-source with ESR 0.7471, while Qwen3-VL-8B/32B reach ESR 0.0000.
• On Task 2 editing, all models reach ESR above 0.99; Gemini-3-Pro leads XDRFR at 0.9405, with open models such as Qwen3-VL-235B at 0.9312.
• On a 96-instruction subset, symbolic XML editing (XDRFR 0.940) outperforms pixel-based editing (DRFR 0.821), with the largest gap on Hard instructions (0.867 vs 0.649).

Surprising findings

• Qwen3-VL-235B scores 0.1519 SCS and 0.3462 ESR on Task 1 despite CodeXQA 0.8224, indicating perception does not translate into valid generation.
• Open-source models that fail image-to-XML generation (e.g., Qwen3-VL-32B, ESR 0.0) still achieve strong Task 2 editing fidelity (XDRFR 0.8873-0.9312).

Limitations

• Limited to Draw.io/mxGraph; not validated on other diagram languages such as Visio, TikZ, or Mermaid.
• Editing focuses on objectively verifiable rule-based edits and does not capture semantic ambiguity or open-ended redesign.
• Executability does not guarantee semantic or topological correctness; SCS depends on a VLM judge whose absolute values are judge-calibrated.

Experimental setup

Datasets: SALT-NLP/Design2Code

Benchmarks: VCG-Bench

Metrics: ESR SCS CodeXQA SigLIP2 similarity XDRFR

Baselines: Gemini-3-Flash Claude-4.5-Sonnet Qwen3-VL-235B Qwen3-VL-32B Qwen3-VL-8B GLM-4.6V Qwen3-Omni-30B Step3 Paper2Any pixel-based editing (GPT-Image)

Builds on: Design2Code PlotCraft DRFR (InFoBench) SigLIP2

Closest papers by meaning

• From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing (CVPR 2025 · similarity 0.89)
• Factuality Matters: When Image Generation and Editing Meet Structured Visuals (ICLR 2026 · similarity 0.87)
• BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing (CVPR 2025 · similarity 0.87)
• JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence (ICLR 2026 · similarity 0.87)
• DrawingsDreamer: A Unified Multi-View Engineering Drawings Generation Model (NeurIPS 2026 · similarity 0.87)
• EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editing (ICLR 2026 · similarity 0.86)