Research map / Large language models / Reasoning and chain of thought

TrimR: Verifier-based Training-Free Thinking Trimming for Efficient Test-Time Scaling

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TrimR, a training-free framework that uses a small pretrained instruction-tuned verifier (7B) to detect and truncate overthinking and underthinking in large reasoning models, with an asynchronous vLLM/Ascend serving system, reducing runtime up to 70% with small accuracy changes.

Problem

Large reasoning models generate redundant thinking (overthinking on easy problems, underthinking/oscillation on hard ones, plus repetition loops), causing high decoding cost and latency in test-time scaling.

Approach

Segment chain-of-thought at reflection tokens, use a lightweight instruction-tuned verifier to check answer existence and consecutive answer equivalence via Yes/No prefill probabilities, and inject gentle or forceful termination prompts into the LRM to stop redundant reasoning, without fine-tuning either model.

Key results

• On MATH500, DeepSeek-R1-Distill-Qwen-32B runtime drops 67.0% (7602s to 2511s) with accuracy rising from 90.4% to 92.4%.
• QwQ-32B on MATH500 runtime falls 29.3% (4413s to 3118s) with accuracy rising from 95.6% to 96.8%.
• Pangu-7B verifier reaches 87.87% chain-pair consistency accuracy versus 86.70% for Qwen2.5-7B-Instruct on annotated MATH500 pairs.

Surprising findings

• Accuracy often improves alongside runtime reduction, e.g. DeepSeek-R1-Distill-Qwen-32B gains 13.2% on GPQA Diamond while runtime drops 70%.
• Underthinking compression alone yields smaller token savings than overthinking compression on QwQ-32B MATH500 (TPR -4.8% vs -12.0%).

Limitations

• Gains reported only on Ascend NPU/vLLM serving setup with large-batch workloads; generalization to other hardware is asserted but not shown.
• Verifier accuracy is only ~87% on manually annotated chain pairs, so consistency errors can occur.
• Pangu Pro MoE shows limited gains on GPQA Diamond because multiple-choice thinking patterns differ.

Experimental setup

Datasets: MATH500 AIME24 AIME25 GPQA Diamond

Benchmarks: MATH500 AIME24 AIME25 GPQA Diamond

Metrics: accuracy runtime TPR TPR-T90 tokens

Baselines: TokenBudget Chain of Draft Dynasor Speculative Rejection ST-BoN

Builds on: test-time scaling chain of draft TokenBudget

Closest papers by meaning

• VeriThinker: Learning to Verify Makes Reasoning Model Efficient (NeurIPS 2025 · similarity 0.91)
• TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.90)
• Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.90)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.89)
• TRIM: Hybrid Inference via Targeted Stepwise Routing in Multi-Step Reasoning Tasks (ICLR 2026 · similarity 0.89)