Research map / Vision-language and multimodal / Visual question answering
ChartR: Evaluating Reasoning Accuracy and Robustness in Chart Question Answering
Xiaojun Chen, Sixiao Luo, Ziqi Liu, Min Yang, Qin Zhang, Liang-Jie Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove numeric annotations from the chart (ablation) | Gemini-2.0-flash · ChartR | ISA / CFA: ISA 88.94 -> 62.50; CFA 54.50 -> 28.00 | Hurt |
| remove numeric annotations from the chart (ablation) | Qwen2.5-VL-7B · ChartR | ISA / CFA: ISA 77.72 -> 54.61; CFA 32.00 -> 15.50 | Hurt |
| add Gaussian noise to the chart (ablation) | Gemini-2.0-flash · ChartR | ISA: 88.94 -> 89.19 | No clear effect |
| add blur to the chart (ablation) | Qwen2.5-VL-7B · ChartR | ISA: 77.72 -> 76.99 | No clear effect |
| add watermark to the chart (ablation) | Phi-4-multimodal-5.6B · ChartR | ISA / CFA: 63.46 -> 60.17; CFA 7.50 -> 5.00 | No clear effect |
| require all reasoning steps correct (CFA) instead of final answer only (FAA) | InternVL2.5-8B · ChartR | accuracy: 43.20 -> 8.90 | Hurt |
| require chained step correctness (CSA) instead of isolated step correctness (ISA) | Gemini-2.0-flash · ChartR | accuracy: 83.01 -> 63.29 | Hurt |
| use chart-specialized models instead of general-purpose models | TinyChart-3B / ChartGemma-2.4B · ChartR | ISA / CFA: ISA 28.29 / CFA 0.30 (TinyChart); ISA 27.78 / CFA 0.00 (ChartGemma) vs Gemini 83.01 / 50.60 | Hurt |
| correct preceding Value Extraction errors | Gemini-2.0-flash · ChartR | recovery rate of VC errors: RecR 95.45% | Improved |
| correct preceding Value Extraction errors | Qwen2.5-VL-7B · ChartR | recovery rate of NC errors: RecR 94.30% | Improved |
Closest papers by meaning
- ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.94)
- InfoChartQA: A Benchmark for Multimodal Question Answering on Infographic Charts (NeurIPS 2025 · similarity 0.91)
- Synthesize Step-by-Step: Tools Templates and LLMs as Data Generators for Reasoning-Based Chart VQA (CVPR 2024 · similarity 0.91)
- Union Is Strength! Unite the Power of LLMs and MLLMs for Chart Question Answering (AAAI 2025 · similarity 0.91)
- ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding (CVPR 2026 · similarity 0.90)
- CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
