Research map / Large language models / Reasoning and chain of thought

TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Process reward models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TaTToo, a tool-integrated generative process reward model for tabular reasoning that is trained via SFT on ~60k step-level table-grounded annotations and then RL (modified GRPO) with tool-grounded reward shaping, improving Best-of-N test-time scaling of DeepSeek-R1-Distill-Qwen-14B.

Problem

Existing general-domain process reward models fail to supervise table-specific reasoning steps (table retrieval and schema interaction), causing performance saturation in test-time scaling for large reasoning models on tabular tasks.

Approach

A table-grounded generative PRM that separately rewards table-operation steps and inner-reasoning steps, integrates tool calls (Python/SQL/DataFrame lookups) into its verification rationales, and is refined with RL using a per-step reward combining label matching, confidence calibration, and tool-grounding.

Key results

• The 8B TaTToo PRM improves DeepSeek-R1-Distill-Qwen-14B by 30.9% at inference across benchmarks, and outperforms Qwen2.5-Math-PRM-72B on Best-of-N (e.g., TB-DA 34.3 vs 32.4 at N=32).
• RL stage raises average accuracy from 72.3% (SFT only) to 78.5%, a reported 10.2% gain.
• Removing tool-grounding term drops TB-DA accuracy by 4.0% at N=32 in ablation.

Surprising findings

• Existing PRMs produce nearly identical rewards for real retrieved sub-tables and randomly replaced sub-tables, indicating they cannot judge retrieval correctness.
• Best-of-N performance of strong PRMs saturates at N>=8 on table tasks.

Limitations

• Relies on expert LLM (DeepSeek-R1, Claude-Opus-4.1) and LLM-as-a-judge labels for much of the curated data, with manual replacement of sub-tables.
• Evaluation uses a single policy model (DeepSeek-R1-Distill-Qwen-14B) in main experiments.
• Theoretical policy-improvement bound relies on idealized NPG assumptions and omits some proof details.

Experimental setup

Datasets: TableInstruct HybridQA ToTTo WikiTQ TableBench WTQ MMQA

Benchmarks: TableBench (NR, FC, DA) WTQ MMQA

Metrics: accuracy under Best-of-N average accuracy under beam search and DVTS

Baselines: Majority Vote LLM-as-a-judge Skywork-PRM-7B Math-Shepherd-PRM-7B Qwen2.5-Math-PRM-7B ThinkPRM GenPRM Qwen2.5-Math-PRM-72B

Builds on: GRPO Math-Shepherd ThinkPRM GenPRM Table-R1

Closest papers by meaning

• From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling (AAAI 2026 · similarity 0.89)
• Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.89)
• Rethinking Reward Models for Multi-Domain Test-Time Scaling (NeurIPS 2026 · similarity 0.89)
• Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning (ICLR 2026 · similarity 0.88)
• Unlocking Multimodal Mathematical Reasoning via Process Reward Model (NeurIPS 2025 · similarity 0.88)
• Know What You Don't Know: Uncertainty Calibration of Process Reward Models (NeurIPS 2025 · similarity 0.88)