Research map / Vision-language and multimodal / Multimodal LLMs

Format Matters: The Robustness of Multimodal LLMs in Reviewing Evidence from Tables and Charts

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Extends SciTabAlign and ChartMimic into table/chart-aligned claim verification datasets and shows 12 open multimodal LLMs verify claims better from tables than from equivalent charts, while humans do not show this gap.

Problem

Whether multimodal LLMs can verify scientific claims consistently when the same experimental evidence is presented as a table versus a chart; existing claim verification datasets do not provide aligned table and chart evidence.

Approach

Construct SciTabAlign+ (70 tables rendered into four chart types for 162 claims) and ChartMimic+ (152 claims on 76 bar/line charts with extracted tables) so each claim has equivalent table and chart evidence, then compare model performance across formats and against human annotators.

Key results

• Across 11 of 12 models, table-only input outperforms chart-only input on SciTabAlign+; e.g., Qwen2.5-VL-72B reaches 86.3 macro-F1 on tables vs 68.5 average on charts.
• Humans score 94.0 (table-only) and 96.0 (chart-only) macro-F1 on a 50-sample subset, showing no format gap.
• Table-chart correlation is low for smaller models (under 8B) and for LLaVA/Llama families, with InternVL3-1B showing negative correlation on SciTabAlign+.

Surprising findings

• Qwen2.5-VL-7B performs better with chart input than table input on ChartMimic+, the only model to do so.
• Adding the chart to table input for LLaVA-v1.6-34B and several smaller models does not help and sometimes hurts relative to table-only input.

Limitations

• SciTabAlign+ is small: 70 tables and 162 claims, with only 50 samples used for the human study.
• Only single-chart scenarios are covered; sub-charts and multi-chart evidence are excluded.
• Only open-weight models evaluated, and only zero-shot CoT prompting used as the main setting.

Experimental setup

Datasets: SciTabAlign ChartMimic

Benchmarks: SciTabAlign+ ChartMimic+

Metrics: macro-F1 Pearson correlation

Baselines: Human annotators

Builds on: SciTab SciTabAlign ChartMimic zero-shot chain-of-thought

Closest papers by meaning

• PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies (ICLR 2026 · similarity 0.90)
• CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (NeurIPS 2024 · similarity 0.89)
• MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.87)
• SO-Bench: A Structural Output Evaluation of Multimodal LLM (CVPR 2026 · similarity 0.87)
• Multimodal Fact-Level Attribution for Verifiable Reasoning (ICML 2026 · similarity 0.86)
• TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning (NeurIPS 2026 · similarity 0.86)