Research map / Large language models / Retrieval and knowledge
OpenTab: Advancing Large Language Models as Open-domain Table Reasoners
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OpenTab, a training-free open-domain table reasoning framework that retrieves tables with BM25, generates simple-to-complex SQL programs with an LLM, reranks tables via generative reranking, and answers with an LLM reader.
Problem
Answering questions and verifying facts over large corpora of structured tables when gold tables are not given, which existing text-oriented retrieval-augmented LLMs and fine-tuned table systems handle poorly.
Approach
Combine a sparse BM25 table retriever with few-shot LLM SQL generation in a simple-to-complex progression, use the SQL execution results plus schema and sampled rows as context for an LLM reader, and rerank retrieved tables by cross-encoder similarity between the query and generated SQL (Generative Reranking & Sequential Reasoning), with no fine-tuning.
Key results
- On Open-WikiTables in the open-domain setting with gpt-3.5-turbo, OpenTab reaches 0.565 at top-10 versus 0.375 for the best baseline (BINDER), a 21.5-point gain.
- On closed-domain WikiTableQuestions, OpenTab reaches 0.641 accuracy, above the fine-tuned OmniTab (0.633) without any fine-tuning.
- BM25 retrieval achieves Recall@5 of 0.832 on Open-WikiTables and 0.919 on FEVEROUS, outperforming non-fine-tuned DPR-BERT, which scores near 0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OpenTab instead of the strongest baseline, LLM-SQL | gpt-3.5-turbo open-domain table reasoning · Open-WikiTables top-10 | accuracy: 0.109 -> 0.565 | Helped |
| Use OpenTab instead of the strongest baseline, BINDER | falcon-180B open-domain table reasoning · Open-WikiTables top-10 | accuracy: 0.227 -> 0.437 | Helped |
| Use OpenTab instead of the strongest baseline, LLM-Answer | gpt-3.5-turbo open-domain table reasoning · FEVEROUS top-10 | accuracy: 0.453 -> 0.508 | Helped |
| Use OpenTab instead of the strongest baseline | closed-domain table QA · WikiTableQuestions | accuracy: 0.633 -> 0.641 | Helped |
| Add Simple-to-Complex prompting (ablation) | OpenTab · Open-WikiTables closed-domain | accuracy: 0.539 -> 0.710 | Helped |
| Add RowSelector (ablation) | OpenTab with Simple-to-Complex prompting removed · Open-WikiTables closed-domain | accuracy: 0.515 -> 0.539 | Helped |
| Add broad-context Reader (ablation) | OpenTab without Simple-to-Complex prompting and RowSelector · Open-WikiTables closed-domain | accuracy: 0.421 -> 0.515 | Helped |
| Combine SQL-basic, SQL-intermediate, and SQL-advanced rather than use an individual SQL program (ablation) | OpenTab Coder · table reasoning ablation | accuracy | Helped |
| Use GRSR rather than Joint Reasoning or Sequential Reasoning (ablation) | OpenTab open-domain reasoning · open-domain table reasoning | accuracy | Helped |
Surprising findings
- Non-fine-tuned DPR-BERT scores nearly zero recall on both datasets, showing poor transferability of dense table retrievers.
- Removing the simple-to-complex SQL strategy drops closed-domain Open-WikiTables accuracy from 0.71 to 0.54.
Limitations
- Evaluated on subsets (2,000 Open-WikiTables validation samples; 323 FEVEROUS claims) due to budget.
- Main results rely on proprietary gpt-3.5-turbo; BINDER was reproduced with gpt-3.5-turbo instead of Codex.
- FEVEROUS evaluation uses a custom metric requiring correct evidence table identification.
Experimental setup
Datasets: Open-WikiTables WikiTableQuestions FEVEROUS Spider
Benchmarks: Open-WikiTables WikiTableQuestions FEVEROUS
Metrics: execution accuracy accuracy (FEVEROUS prediction plus correct evidence table) Recall@k
Baselines: LLM-Answer LLM-SQL BINDER DATER DPR-BERT T5-3B Tapex TaCube
Builds on: BINDER DATER BM25
Closest papers by meaning
- Table as a Modality for Large Language Models (NeurIPS 2025 · similarity 0.89)
- TableRAG: Million-Token Table Understanding with Language Models (NeurIPS 2024 · similarity 0.89)
- TableBench: A Comprehensive and Complex Benchmark for Table Question Answering (AAAI 2025 · similarity 0.89)
- NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables (NeurIPS 2025 · similarity 0.89)
- Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding (ICLR 2024 · similarity 0.88)
- ConTextTab: A Semantics-Aware Tabular In-Context Learner (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.