Research map / Multi-agent systems and games / LLM agents
How Far Can LLM Agents Reason with Tables? Benchmarking Multi-Turn Agentic Table Question Answering in the Wild
Jingwang Huang, Jie Zhang, Haoyang Zeng, Changzai Pan, Xianjie Wu, Guanting Dong, Jiaheng Liu, Wei Zhang, Mingyu Zheng, Chunxiao Liu, Kaiwen Wei, Jiang Zhong and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TableAgent-Bench, a benchmark for evaluating LLM agents on multi-turn, tool-using table question answering in realistic settings.
Problem
Existing TableQA benchmarks treat the task as passive, single-turn understanding and cannot evaluate autonomous reasoning and tool-call trajectories in multi-turn scenarios.
Approach
A large-scale benchmark that frames TableQA as multi-turn agentic interaction, evaluating reasoning and tool-call trajectories rather than only single-turn answers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable thinking mode | Kimi-K2 · TableAgent-Bench | information coverage: +12.1% IC over non-thinking | Improved |
| enable thinking mode | Claude-Haiku-4-5 · TableAgent-Bench | information coverage: 41.00 -> 51.10 | Improved |
| enable thinking mode | DeepSeek-V3.2 · TableAgent-Bench | information coverage: 27.50 -> 46.80 | Improved |
| enable thinking mode | Qwen3-8B · TableAgent-Bench | information coverage: 2.80 -> 2.75 | No clear effect |
| add exact-matching and semantic-retrieval tools to code execution (ablation) | Claude-Haiku-4-5 · toolset ablation | information coverage: 46.20 -> 53.40 | Improved |
| add exact-matching and semantic-retrieval tools to code execution (ablation) | DeepSeek-V3.2 · toolset ablation | table relevance recall: 65.1 -> 78.2 | Improved |
| add exact-matching tool to code execution (ablation) | DeepSeek-V3.2 · toolset ablation | information coverage: 43.0 -> 41.50 | Hurt |
| add table processing tools to CE+EM+SR (ablation) | DeepSeek-V3.2 · toolset ablation | information coverage: 44.90 -> 46.80 | Improved |
| add table processing tools to CE+EM+SR (ablation) | Mimo-V2-Flash · toolset ablation | information coverage: 45.00 -> 45.60 | Improved |
| add table processing tools to CE+EM+SR (ablation) | Claude-Haiku-4-5 · toolset ablation | information coverage: 53.40 -> 51.10 | Hurt |
| increase interaction steps at fixed 128k context, or context length at fixed 70 steps (ablation) | table agents · context/step analysis | performance | Improved |
| higher table recall and precision | table agents · TableAgent-Bench | information coverage | Improved |
Experimental setup
Benchmarks: TableAgent-Bench
Closest papers by meaning
- AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering (NeurIPS 2026 · similarity 0.89)
- TableBench: A Comprehensive and Complex Benchmark for Table Question Answering (AAAI 2025 · similarity 0.89)
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.88)
- MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.88)
- \$OneMillion-Bench: How Far are Language Agents from Human Experts? (NeurIPS 2026 · similarity 0.88)
- Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
