Research map / Large language models / Reasoning and chain of thought

T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Tool-integrated Verification (T1), a two-stage verifier in which code-based or retrieval-based tool checks filter candidate solutions before a distilled small LM reward model (PRM or GenRM) scores them, enabling small models to verify under best-of-N scaling.

Problem

Small language models struggle to verify candidate solutions for test-time compute scaling, especially on calculation and fact-checking steps that require memorization.

Approach

Offload memorization-heavy verification (arithmetic via code interpreter, fact-checking via retrieved documents) to external tools as a binary filter before a reward-model score, with both stages distilled from a larger teacher; theory argues tools reduce the memorization burden and improve best-of-N accuracy.

Key results

• On MATH500 with PRM, Llama-3.2-1B with ToolV outperforms the Llama-3.1-8B model under best-of-64 scaling.
• On MATH500 Llama-3.2-1B, GenRM + ToolV reaches 86.99% verification accuracy versus 80.91% for GenRM alone.
• On MATH500 with GenRM at N=64, GenRM + ToolV reaches 50.6 versus 45.4 for one-stage GenRM.

Surprising findings

• ToolV alone already surpasses PRM on MATH500 and GenRM on GSM8K according to the paper's ablation.
• Under a compute-normalized x-axis, the ToolV overhead corresponds to only about a 1.14x larger verifier, and the gains persist.

Limitations

• ToolV acts only as a rejection filter and cannot recover correct solutions wrongly rejected (false negatives).
• Evaluated only on best-of-N parallel scaling, not step-level search or sequential scaling.
• Tool verification accuracy depends on generated code quality; recall of ToolV is low, and gains on Geometry and Level 5 problems are weak.

Experimental setup

Datasets: MATH training set GSM8K training set MMLU-Pro subset (Health, Economics, History) Wikipedia abstracts (BM25 retrieval)

Benchmarks: MATH500 GSM8K MMLU-Pro subset

Metrics: weighted best-of-N accuracy verification accuracy precision recall F1

Baselines: Majority Voting Zero-shot GenRM Distilled PRM Distilled GenRM Themis VersaPRM

Builds on: Themis Math-Shepherd GenRM-CoT Program-aided language models LoRA

Closest papers by meaning

• Rethinking Optimal Verification Granularity for Compute-Efficient Test-Time Scaling (NeurIPS 2025 · similarity 0.92)
• Variation in Verification: Understanding Verification Dynamics in Large Language Models (ICLR 2026 · similarity 0.91)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.89)
• Optimal Aggregation of LLM and PRM Signals for Efficient Test-Time Scaling (ICLR 2026 · similarity 0.89)
• Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck (NeurIPS 2026 · similarity 0.89)
• CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.88)