Research map / Vision-language and multimodal / Multimodal LLMs
SynTab-LLaVA: Enhancing Multimodal Table Understanding with Decoupled Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Decouples multimodal table understanding data synthesis into table-code-to-image rendering and LLM-based Q&A generation from code, producing the 1.8M-sample SynTab dataset and the SynTab-LLaVA model with a hybrid high/low-resolution encoder.
Problem
Multimodal table understanding (MTU) is limited by scarce training data; annotating via humans is costly and MLLM-based synthesis from table images is expensive and prone to hallucination.
Approach
Render table images from code (HTML/LaTeX/Markdown) and generate Q&A pairs by prompting an LLM with the table code rather than the image, decoupling the two steps; pair with a hybrid encoder combining a high-res ConvNeXt and low-res ViT-L/14 features concatenated along channels.
Key results
- SynTab-LLaVA reaches 88.30 on TABMWP and 51.94 on TAT-QA, and achieves SOTA on 21 of 24 in-domain and out-of-domain benchmarks.
- The 636K-image, 1.8M-sample SynTab dataset was synthesized for under $200 versus about $12,000 for 2,000 TableBench pairs with human annotation.
- In ablation, the hybrid ConvNeXt + ViT encoder improves average TSU from 44.46 to 58.56 relative to ConvNeXt alone (+14.10 vs baseline), with SynTab-Pre and SynTab-SFT together adding +4.54 TSU and +4.86 TQA over the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SynTab-Pre during pre-training (ablation) | SynTab-LLaVA · TSU | score: +1.69 | Helped |
| Add SynTab-Pre during pre-training (ablation) | SynTab-LLaVA · TQA | score: +0.49 | Helped |
| Add SynTab-Pre during pre-training (ablation) | SynTab-LLaVA · TFV | score: +0.07 | Helped |
| Add SynTab-Pre during pre-training (ablation) | SynTab-LLaVA · T2T | score: +0.50 | Helped |
| Add SynTab-SFT during fine-tuning (ablation) | SynTab-LLaVA · TQA | score: +3.97 | Helped |
| Add SynTab-SFT during fine-tuning (ablation) | SynTab-LLaVA · TFV | score: +2.45 | Helped |
| Add SynTab-SFT during fine-tuning (ablation) | SynTab-LLaVA · T2T | score: +1.28 | Helped |
| Add SynTab-SFT during fine-tuning (ablation) | SynTab-LLaVA · TSU | score: +4.13 | Helped |
| Use both SynTab-Pre and SynTab-SFT (ablation) | SynTab-LLaVA · TQA | score: +4.86 | Helped |
| Use both SynTab-Pre and SynTab-SFT (ablation) | SynTab-LLaVA · TFV | score: +3.31 | Helped |
| Use both SynTab-Pre and SynTab-SFT (ablation) | SynTab-LLaVA · T2T | score: +1.47 | Helped |
| Use both SynTab-Pre and SynTab-SFT (ablation) | SynTab-LLaVA · TSU | score: +4.54 | Helped |
Surprising findings
- AIT-QA accuracy is low (28.57) for SynTab-LLaVA, below Qwen2-VL and IX2.5, which the authors attribute to square padding of very wide tables.
Limitations
- SynTab contains some erroneous annotations that may hurt training.
- Performance degrades on tables with rotation, occlusion, deformation, or perspective distortion typical of industrial scenarios.
- Padding to a square based on the longest edge hurts very wide tables such as AIT-QA.
Experimental setup
Datasets: FinTabNet TableLLama MMTab LLaVA1.5-Pre WTQ FeTaQA HiTab TAT-QA TABMWP AIT-QA TabMCQ TabFact
Benchmarks: WTQ FeTaQA HiTab TAT-QA TABMWP AIT-QA TabMCQ TabFact InfoTabs PubHealthTab ToTTo HiTab T2T
Metrics: accuracy BLEU F1 TEDS row/column accuracy
Baselines: Table-LLaVA 7B Table-LLaVA 13B Qwen2-VL 7B GPT-4V InternVL2 8B TabPedia IX2.5 Ovis1.5
Builds on: LLaVA-1.5 Table-LLaVA ConvNeXt ViT-L/14
Closest papers by meaning
- TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding (ICLR 2026 · similarity 0.89)
- Multimodal Tabular Reasoning with Privileged Structured Information (NeurIPS 2025 · similarity 0.88)
- TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy (NeurIPS 2024 · similarity 0.88)
- Table as a Modality for Large Language Models (NeurIPS 2025 · similarity 0.88)
- NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables (NeurIPS 2025 · similarity 0.88)
- TabFlash: Efficient Table Understanding with Progressive Question Conditioning and Token Focusing (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.