Research map / Large language models / Retrieval and knowledge
Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Chain-of-Table prompts LLMs to iteratively plan and apply tabular operations (add column, select row/column, group by, sort by), evolving the table into a reasoning chain before a final answer query.
Problem
Table understanding with LLMs via prompting: existing chain-of-thought and program-aided methods represent intermediate reasoning as free-form text or code, which handles complex tables poorly.
Approach
Use the evolving intermediate table itself as the thought in the reasoning chain: the LLM dynamically selects the next atomic table operation (planned from the current table, question, and operation history) and generates its arguments, which are executed programmatically to transform the table before a final query.
Key results
- Chain-of-Table with PaLM 2 reaches 67.31% on WikiTQ and 86.61% on TabFact, improving over the second-best method Dater (61.48%, 84.63%).
- With GPT-3.5, Chain-of-Table achieves 59.94% on WikiTQ and 80.20% on TabFact, gains of +3.20 and +1.03 over the second-best methods.
- On FeTaQA with PaLM 2, Chain-of-Table gets BLEU 32.61 versus 29.47 for Dater; it needs at most 25 generated samples per question versus 100 for Dater and 50 for Binder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Chain-of-Table instead of the strongest baseline | PaLM 2 · TabFact | accuracy: 84.63 -> 86.61 | Helped |
| use Chain-of-Table instead of the strongest baseline | PaLM 2 · WikiTQ | denotation accuracy: 61.48 -> 67.31 | Helped |
| use Chain-of-Table instead of the strongest baseline | GPT 3.5 · TabFact | accuracy: 79.17 -> 80.20 | Helped |
| use Chain-of-Table instead of the strongest baseline | GPT 3.5 · WikiTQ | denotation accuracy: 56.74 -> 59.94 | Helped |
| use Chain-of-Table instead of the strongest baseline | LLaMA 2 · TabFact | accuracy: 65.12 -> 67.24 | Helped |
| use Chain-of-Table instead of the strongest baseline | LLaMA 2 · WikiTQ | denotation accuracy: 41.44 -> 42.61 | Helped |
| use Chain-of-Table instead of Dater | PaLM 2 · FeTaQA | BLEU: 29.47 -> 32.61 | Helped |
| remove add-column operation (adding it back) (ablation) | Chain-of-Table · TabFact | accuracy: 85.23 -> 86.61 | Helped |
| remove select-column operation (adding it back) (ablation) | Chain-of-Table · TabFact | accuracy: 82.61 -> 86.61 | Helped |
| remove select-row operation (adding it back) (ablation) | Chain-of-Table · TabFact | accuracy: 82.21 -> 86.61 | Helped |
| remove group-by operation (adding it back) (ablation) | Chain-of-Table · WikiTQ | accuracy: 61.88 -> 67.31 | Helped |
| remove sort-by operation (adding it back) (ablation) | Chain-of-Table · TabFact | accuracy: 86.21 -> 86.61 | Helped |
Surprising findings
- Vanilla Chain-of-Thought decreases WikiTQ performance over End-to-End QA with PaLM 2 (60.43 vs 60.59).
- ROUGE metrics on FeTaQA give low scores even for answers judged correct, so they are insensitive to improvements under in-context learning.
Limitations
- Only five atomic operations are evaluated; extension to other operations is left for future work.
- Results on the open-source LLaMA 2 model are with a 17B chat variant and rely on the same prompting setup; the Codex-based Dater comparison was not reproduced.
- ROUGE-based FeTaQA evaluation shows marginal gains and may not reflect answer correctness.
Experimental setup
Datasets: WikiTQ TabFact FeTaQA
Benchmarks: WikiTQ TabFact FeTaQA
Metrics: denotation accuracy binary classification accuracy BLEU ROUGE-1 ROUGE-2 ROUGE-L
Baselines: End-to-End QA Few-Shot QA Chain-of-Thought Text-to-SQL Binder Dater
Builds on: Chain-of-Thought Binder Dater Text-to-SQL
Closest papers by meaning
- TableBench: A Comprehensive and Complex Benchmark for Table Question Answering (AAAI 2025 · similarity 0.90)
- Accurate and Regret-Aware Numerical Problem Solver for Tabular Question Answering (AAAI 2025 · similarity 0.88)
- NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables (NeurIPS 2025 · similarity 0.88)
- Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources (ICLR 2024 · similarity 0.88)
- OpenTab: Advancing Large Language Models as Open-domain Table Reasoners (ICLR 2024 · similarity 0.88)
- Multimodal Tabular Reasoning with Privileged Structured Information (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.