Research map / Vision-language and multimodal / Multimodal LLMs

ChartGalaxy: A Dataset for Infographic Chart Understanding and Generation

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ChartGalaxy, a million-scale dataset of real and programmatically synthesized infographic charts paired with data tables, and shows gains for VQA fine-tuning, a code generation benchmark, and example-based chart generation.

Problem

LVLMs struggle with infographic charts, whose rich visual design and layouts differ from plain charts, and existing infographic datasets are small and generating high-quality infographic charts remains hard.

Approach

An inductive pipeline extracts chart types, variations, and layout templates from real infographics (with a human-in-the-loop detector expansion) and uses them to programmatically render synthetic infographic charts paired with data tables, with layout chosen by maximizing ink ratio.

Key results

• Fine-tuning InternVL3-8B on ChartGalaxy data raises ChartQAPro from 38.15 to 44.13 and overall accuracy on the independent evaluation set from 53.20 to 80.07.
• On the code generation benchmark, Gemini-2.5-Pro achieves the top overall score of 85.21 among 17 LVLMs, while the best open-source model Llama-4-Maverick-17B scores 61.29.
• In a 16-expert user study, the proposed example-based method scores 4.63 vs 2.10 for GPT-Image-1 on fidelity, with p < 0.01.

Surprising findings

• Synthetic-only fine-tuning on InternVL3-8B reaches 76.08 on the evaluation set, far above real-only fine-tuning (55.74), while real-only helps more on ChartQAPro.

Limitations

• Dataset focuses on single-chart infographics and does not cover multi-chart narratives.
• Real-chart geographic and domain distribution skews toward North America and Europe and sources like Visual Capitalist and Statista.
• GPT-4o is used as the judge for high-level code generation similarity.

Experimental setup

Datasets: ChartGalaxy VizNet InfographicVQA ChartQAPro

Benchmarks: InfographicVQA ChartQAPro ChartGalaxy evaluation set ChartGalaxy code generation benchmark

Metrics: relaxed accuracy ANLS exact match execution rate low-level score high-level score overall score Likert fidelity/aesthetics/creativity grid coverage

Baselines: GPT-Image-1 GPT-4.1 Gemini-2.5-Pro Claude-3.7-Sonnet Qwen2.5-VL-72B InternVL3-78B Llama-4-Maverick-17B InfographicVQA ChartQAPro

Builds on: InternImage DINO LTSim DreamSim ChartQAPro

Closest papers by meaning

• ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding (CVPR 2026 · similarity 0.90)
• InfoChartQA: A Benchmark for Multimodal Question Answering on Infographic Charts (NeurIPS 2025 · similarity 0.88)
• EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding (AAAI 2025 · similarity 0.88)
• ChartE$^{3}$: A Comprehensive Benchmark for End-to-End Chart Editing (ICML 2026 · similarity 0.87)
• ChartSketcher: Reasoning with Multimodal Feedback and Reflection for Chart Understanding (NeurIPS 2025 · similarity 0.87)
• CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (NeurIPS 2024 · similarity 0.86)