Research map / Vision-language and multimodal / Multimodal LLMs
Multimodal Table Understanding with Difficulty-aware Reinforcement Learning
Chaohu Liu, Haoyu Cao, YongXiang Hua, Linli Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a difficulty-aware reinforcement learning approach for multimodal table understanding that accounts for table structural complexity (merged-cell ratio, total cell count).
Problem
Multimodal table understanding, which integrates cell text, tabular structure, and visual presentation, is challenging because structural complexity (e.g., ratio of merged cells, total cell count) impedes existing models.
Approach
Quantifies table difficulty via intrinsic structural properties (merged-cell ratio and total number of cells) and uses difficulty-aware reinforcement learning to train on tables of varying complexity. Details beyond the abstract are not stated here.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Stage 2 RL after Stage 1 perception training (vs Stage 1 only) (ablation) | MM-Table-R1-3B · HiTab | accuracy: 66.83 -> 76.14 | Improved |
| add Stage 1 table reconstruction RL before Stage 2 on TabFact (ablation) | MM-Table-R1-3B · TabFact | accuracy: 83.53 -> 87.13 | Improved |
| add Stage 1 table reconstruction RL before Stage 2 on ToTTo (ablation) | MM-Table-R1-3B · ToTTo | accuracy: 55.32 -> 78.67 | Improved |
| add Stage 1 table reconstruction RL before Stage 2 on TableVQA (ablation) | MM-Table-R1-3B · TableVQA | accuracy: 72.06 -> 78.60 | Improved |
| add Stage 1 table reconstruction RL before Stage 2 on HiTab (ablation) | MM-Table-R1-3B · HiTab | accuracy: 71.21 -> 76.14 | Improved |
| use RL Stage 1 instead of SFT for perception (ablation) | MM-Table-R1-3B · TableVQA | accuracy: 73.93 -> 78.60 | Improved |
| use RL Stage 1 instead of SFT for perception (ablation) | MM-Table-R1-3B · ToTTo | accuracy: 63.61 -> 78.67 | Improved |
| use RL Stage 1 instead of SFT for perception (ablation) | MM-Table-R1-3B · TabFact | accuracy: 82.19 -> 87.13 | Improved |
| SFT training on Qwen2.5-VL-3B base | Qwen2.5-VL-3B · TableVQA | accuracy: 63.94 -> 62.73 | Hurt |
| SFT training on Qwen2.5-VL-3B base | Qwen2.5-VL-3B · HiTab | accuracy: 63.42 -> 67.91 | Improved |
| Stage 1 only on base model | Qwen2.5-VL-3B · ToTTo | accuracy: 48.61 -> 78.71 | Improved |
| Stage 2 only on base model | Qwen2.5-VL-3B · TableVQA | accuracy: 63.94 -> 72.06 | Improved |
Closest papers by meaning
- TableMix: Enhancing Multimodal Table Reasoning in MLLMs from a Data-Centric Perspective (CVPR 2026 · similarity 0.92)
- TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding (ICLR 2026 · similarity 0.90)
- Multimodal Tabular Reasoning with Privileged Structured Information (NeurIPS 2025 · similarity 0.89)
- Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance (ICML 2026 · similarity 0.89)
- TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy (NeurIPS 2024 · similarity 0.88)
- NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
