Research map / Vision-language and multimodal / Multimodal LLMs
TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TabPedia, a large vision-language model that unifies table detection, structure recognition, table querying and table QA via a concept synergy mechanism using learnable meditative tokens, and introduces the ComTQA benchmark.
Problem
Visual table understanding is fragmented into task-specific architectures for table detection, structure recognition, querying and QA, and existing LVLMs struggle with two-dimensional table structure and high-resolution table content.
Approach
Abstracts VTU tasks and dual-resolution visual embeddings as concepts and couples them through learnable meditative tokens appended to the LLM input, with a coordinate-based table structure representation (five object classes) that lets the LLM output detection-style results without NMS.
Key results
- TabPedia reaches 98.4 F1 on table detection at IoU@0.75 without NMS, matching DETR-based TATR (98.4) and beating Faster R-CNN TATR (89.5).
- On TSR, TabPedia achieves 95.11 S-TEDS on FinTabNet and 95.41 on PubTabNet, exceeding OmniParser (91.55 and 90.45).
- On ComTQA, TabPedia reaches 53.5 accuracy versus 29.3 for Gemini Pro and 27.2 for GPT-4V; it also gets 47.8 on WTQ, the best among compared LVLMs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add meditative tokens (ablation) | TabPedia · PubTab1M-Det | precision: 93.5 -> 98.5 | Helped |
| Add meditative tokens (ablation) | TabPedia · FinTabNet | S-TEDS: 92.17 -> 95.11 | Helped |
| Add meditative tokens (ablation) | TabPedia · WTQ | accuracy: 43.2 -> 47.8 | Helped |
| Use both high- and low-resolution vision encoders (ablation) | TabPedia · PubTab1M-Det | precision: 96.5 -> 98.5 | Helped |
| Use both high- and low-resolution vision encoders (ablation) | TabPedia · FinTabNet | S-TEDS: 93.6 -> 95.11 | Helped |
| Use both high- and low-resolution vision encoders (ablation) | TabPedia · WTQ | accuracy: 44.9 -> 47.8 | Helped |
| Add the high-resolution vision encoder to the low-resolution encoder (ablation) | TabPedia · PubTab1M-Det | precision: 86.2 -> 98.5 | Helped |
| Add the high-resolution vision encoder to the low-resolution encoder (ablation) | TabPedia · FinTabNet | S-TEDS: 81.3 -> 95.11 | Helped |
| Add the high-resolution vision encoder to the low-resolution encoder (ablation) | TabPedia · WTQ | accuracy: 24.7 -> 47.8 | Helped |
| Unfreeze the low-resolution vision encoder during training (ablation) | TabPedia · PubTab1M-Det | precision: 98.5 -> 98.4 | Hurt |
| Unfreeze the low-resolution vision encoder during training (ablation) | TabPedia · FinTabNet | S-TEDS: 95.11 -> 95.11 | No clear effect |
| Unfreeze the low-resolution vision encoder during training (ablation) | TabPedia · WTQ | accuracy: 47.8 -> 46.4 | Hurt |
Surprising findings
- Adding meditative tokens improved TD precision from 93.5 to 98.5 and WTQ accuracy from 43.2 to 47.8.
- Unfreezing the low-resolution encoder gave no significant gain while increasing training time, so it was kept frozen.
Limitations
- Cannot accurately parse twisted or distorted tables because structure is represented by rectangular boxes.
- Evaluated TQA only on table-dominant images; cannot answer questions directly from full document images.
- Slower autoregressive decoding than parallel detectors such as DETR and Faster R-CNN.
Experimental setup
Datasets: PubTab1M FinTabNet PubTabNet WikiTableQuestions TabFact PubTab1M-Syn
Benchmarks: PubTab1M-Det FinTabNet PubTabNet PubTab1M-Str PubTab1M-Syn WTQ TabFact ComTQA
Metrics: precision recall F1 at IoU@0.75 S-TEDS GriTS accuracy
Baselines: TATR Donut EDD OmniParser TextMonkey Monkey CogAgent DocOwl 1.5 GPT-4V Gemini Pro XComposer2
Builds on: Donut Vicuna CLIP TATR OmniParser Memory Transformer
Closest papers by meaning
- TABLET: A Large-Scale Dataset for Robust Visual Table Understanding (ICLR 2026 · similarity 0.92)
- TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding (ICLR 2026 · similarity 0.91)
- Multimodal Tabular Reasoning with Privileged Structured Information (NeurIPS 2025 · similarity 0.89)
- TabFlash: Efficient Table Understanding with Progressive Question Conditioning and Token Focusing (AAAI 2026 · similarity 0.89)
- TRivia: Self-supervised Fine-tuning of Vision-Language Models for Table Recognition (CVPR 2026 · similarity 0.88)
- SynTab-LLaVA: Enhancing Multimodal Table Understanding with Decoupled Synthesis (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.