Research map / Large language models / Retrieval and knowledge
TableRAG: Million-Token Table Understanding with Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TableRAG, a retrieval-augmented framework that uses LM-generated query expansion with schema and cell retrieval (frequency-capped cell budget) to let program-aided LM agents reason over million-cell tables.
Problem
LM-based table understanding does not scale to very large tables: full tables exceed context limits, suffer lost-in-the-middle degradation, and cost grows with table size.
Approach
Instead of encoding entire rows/columns, TableRAG separately retrieves column schemas and distinct column-cell pairs (top-B by frequency) using LM-expanded queries, and passes only the top-K retrieved schema and cells to a ReAct-style program-aided solver, making prompt size independent of table size.
Key results
- On ArcadeQA with GPT-3.5 Turbo, TableRAG reaches 49.2% accuracy vs 43.1% for ReadSchema and 4.6% for ReadTable.
- On BirdQA with GPT-3.5 Turbo, TableRAG reaches 45.5% vs 40.3% for ReadSchema.
- On WikiTableQA, TableRAG achieves 57.03 accuracy, above Binder (56.74) and Dater (52.81).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TableRAG instead of ReadSchema | GPT-3.5 Turbo table QA · ArcadeQA | accuracy: 43.1 -> 49.2 | Helped |
| Use TableRAG instead of ReadSchema | GPT-3.5 Turbo table QA · BirdQA | accuracy: 40.3 -> 45.5 | Helped |
| Use TableRAG instead of ReadSchema | Gemini 1.0 Pro table QA · ArcadeQA | accuracy: 30.8 -> 42.3 | Helped |
| Use TableRAG instead of ReadSchema | Gemini 1.0 Pro table QA · BirdQA | accuracy: 31.8 -> 44.2 | Helped |
| Use TableRAG instead of ReadSchema | Mistral Nemo table QA · ArcadeQA | accuracy: 32.3 -> 46.2 | Helped |
| Use TableRAG instead of RowColRetrieval | Mistral Nemo table QA · BirdQA | accuracy: 36.4 -> 45.1 | Helped |
| Replace BM25 with embedding-based retrieval (ablation) | TableRAG · ArcadeQA | accuracy: 37.7 -> 49.2 | Helped |
| Replace hybrid retrieval with embedding-based retrieval (ablation) | TableRAG · BirdQA | accuracy: 44.5 -> 45.5 | Helped |
| Add cell retrieval while keeping all schemas (ablation) | TableRAG · ArcadeQA | accuracy: 43.1 -> 48.5 | Helped |
| Add cell retrieval while keeping all schemas (ablation) | TableRAG · BirdQA | accuracy: 40.3 -> 42.5 | Helped |
| Retrieve schemas instead of using all schemas, without cell retrieval (ablation) | TableRAG · ArcadeQA | accuracy: 43.1 -> 46.9 | Helped |
| Retrieve schemas instead of using all schemas, without cell retrieval (ablation) | TableRAG · BirdQA | accuracy: 40.3 -> 44.8 | Helped |
Surprising findings
- Increasing the number of top retrieval results K does not consistently improve accuracy, as longer contexts worsen lost-in-the-middle effects.
- RowColRetrieval performance declines as the cell encoding budget grows, attributed to noisier embeddings from longer column sequences.
Limitations
- Worst-case cell retrieval complexity is O(NM) when all cell values are distinct, risking loss of information despite frequency truncation.
- Evaluation limited to QA and verification tasks due to scarcity of large-scale table benchmarks.
- Compares against baselines using the same solver rather than latest state-of-the-art table reasoning methods.
Experimental setup
Datasets: ArcadeQA BirdQA Synthetic TabFact TabFact WikiTableQA
Benchmarks: ArcadeQA BirdQA Synthetic TabFact WikiTableQA
Metrics: exact-match accuracy retrieval recall retrieval precision retrieval F1
Baselines: ReadTable ReadSchema RandRowSampling RowColRetrieval TaBERT Text-to-SQL Binder Dater
Builds on: ReAct PyReAct Dater Binder Chain-of-Table
Closest papers by meaning
- TableBench: A Comprehensive and Complex Benchmark for Table Question Answering (AAAI 2025 · similarity 0.89)
- OpenTab: Advancing Large Language Models as Open-domain Table Reasoners (ICLR 2024 · similarity 0.89)
- NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables (NeurIPS 2025 · similarity 0.88)
- MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark (NeurIPS 2025 · similarity 0.88)
- Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data (ICLR 2025 · similarity 0.87)
- Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queries (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.