Research map / Large language models / Reasoning and chain of thought

Test-time Prompt Intervention

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Test-time Prompt Intervention (PI), a training-free framework that inserts trigger prompts at high-entropy step boundaries and selects branches by a combined perplexity and reasoning-depth score, to shorten chain-of-thought and reduce hallucination in reasoning models.

Problem

Large reasoning models produce redundant chains of thought (repetitive verification, unnecessary reasoning shifts) because post-training relies on outcome rewards and process supervision is hard to scale; the paper asks whether test-time intervention can regulate reasoning paths to be more concise a

Approach

At step boundaries where the first-token entropy is high, the method forks several branches using prompt triggers (progression, summary, verification, conclusion, or the model's natural continuation) and picks the branch with a joint score of inverse perplexity and a Jensen-Shannon-based Reasoning Depth Score computed from early-layer versus final-layer logit distributions.

Key results

• On Qwen3-8B, PI-π^d(p,s) reaches 75.8 overall accuracy (vs 74.5 vanilla) at 50.2% of the original token count.
• On DeepSeek-R1-Distill-Qwen-14B, PI reaches 72.9 overall accuracy at 50.4% compression rate, outperforming DEER (71.6, 65.0%).
• On GSM-NoOp with DeepSeek-R1-Distill-Qwen-14B, PI-π^d(p,s) reaches 85.0% accuracy vs 82.5% vanilla while using 677 tokens vs 1998.

Surprising findings

• Removing verification steps (NoVerify post-processing) raised accuracy while cutting token cost in the preliminary study.
• Adding a conclusion branch shortens reasoning further but significantly degrades accuracy on challenging datasets.

Limitations

• Evaluated only on Qwen3 and DeepSeek-R1-Distill models with hand-designed trigger prompts and hyperparameters (alpha=0.6, entropy threshold 0.3).
• Static interventions hurt accuracy on hard problems, and the conclusion branch degrades challenging-dataset accuracy.
• Multi-branch expansion adds inference cost that is not fully characterized in the main text.

Experimental setup

Datasets: GSM8K MATH-500 AMC 2023 OlympiadBench GPQA Diamond Minerva GSM-NoOp TruthfulQA

Benchmarks: GSM8K MATH-500 AMC 2023 OlympiadBench GPQA Diamond Minerva GSM-NoOp TruthfulQA

Metrics: accuracy token count compression rate

Baselines: Vanilla NoThinking NOWAIT DEER

Builds on: s1 budget forcing DEER NOWAIT DeepSeek-R1

Closest papers by meaning

• Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning (NeurIPS 2025 · similarity 0.89)
• Does Thinking More Always Help? Mirage of Test-Time Scaling in Reasoning Models (NeurIPS 2025 · similarity 0.88)
• Rethinking Thinking Tokens: LLMs as Improvement Operators (ICML 2026 · similarity 0.88)
• Bag of Tricks for Inference-time Computation of LLM Reasoning (NeurIPS 2025 · similarity 0.88)
• When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs (NeurIPS 2025 · similarity 0.88)
• Eliciting Reasoning in Language Models with Cognitive Tools (NeurIPS 2025 · similarity 0.87)