Research map / Vision-language and multimodal / Multimodal LLMs

ChartE$^{3}$: A Comprehensive Benchmark for End-to-End Chart Editing

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ChartE3, a benchmark of ~1,200 image-to-image chart editing instructions evaluated with objective image metrics and GPT-based correctness/consistency scores across closed and open image editing models.

Problem

Chart editing is usually done through code-mediated pipelines, which decouple visual perception from editing; end-to-end image-based chart editing lacks a rigorous benchmark.

Approach

Evaluate chart editing directly as image-to-image transformation with reference edited images built from a chart-code-instruction pipeline, covering 12 local and global editing tasks, and score with both objective visual metrics and GPT-5.1 judged correctness and consistency.

Key results

• Nano Banana achieves the best results, with SSIM 0.83, PSNR 17.53, LPIPS 0.23, and GPT correctness/consistency of 4.18/4.65, while open-source models such as OmniGen2 reach only 0.55 SSIM and 8.70 PSNR.
• Global editing is much harder than local editing: Nano Banana scores 3.42 correctness on Order and 3.59 on Filtering, versus 4.82 on Color.
• GPT-based scores agree with human rankings at 0.79 pairwise accuracy for correctness and 0.87 for consistency on 140 samples.

Surprising findings

• Code-mediated GPT-5.1 (93.14% execution) underperforms end-to-end Nano Banana on most objective and subjective metrics.
• GPT-Image-1.5 scores lower than Nano Banana on Filtering and Order, dropping to 2.44 correctness on Order.

Limitations

• Dataset scale (~1,200 samples) is modest and may miss rare chart styles and long-tail edits.
• Data construction relies on proprietary APIs (Gemini-2.5-Pro, GPT), affecting reproducibility.
• Only static 2D charts are covered; interactive dashboards and maps are excluded.

Experimental setup

Datasets: ChartBench Chart2Code ChartX ChartM3

Benchmarks: ChartE3

Metrics: SSIM PSNR CLIP similarity DINO similarity LPIPS GPT correctness score GPT consistency score

Baselines: InstructPix2Pix OmniGen2 Step1X-Edit BAGEL Qwen-Image-Edit GPT-Image-1.5 Nano Banana GPT-5.1 (w/ code)

Builds on: ChartX ChartM3 Chart2Code ChartBench

Closest papers by meaning

• ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding (CVPR 2026 · similarity 0.89)
• ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding (ICLR 2025 · similarity 0.87)
• ChartGalaxy: A Dataset for Infographic Chart Understanding and Generation (ICLR 2026 · similarity 0.87)
• CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (NeurIPS 2024 · similarity 0.86)
• Factuality Matters: When Image Generation and Editing Meet Structured Visuals (ICLR 2026 · similarity 0.86)
• EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding (AAAI 2025 · similarity 0.86)