Research map / Large language models / Reasoning and chain of thought

Echoes as Anchors: Probabilistic Costs and Attention Refocusing in LLM Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes the spontaneous "Echo of Prompt" in large reasoning models, proposes ED-SFT (fine-tuning to instill echo-then-reason) and training-free Echoic Prompting, and attributes gains to mid-layer attention refocusing on the answer prefix.

Problem

Large reasoning models often spontaneously restate the user's question at the start of their reasoning trace, but it is unclear whether this echo is a superfluous artifact or a functional test-time compute strategy.

Approach

Frames echo removal as rejection-based conditioning, defines the Echo Likelihood Gap as a computable proxy for the probabilistic cost of an echo, and exploits the echo via distillation-based SFT and a training-free re-prompting method that re-inserts the question mid-trace.

Key results

• ED-SFT on Qwen3-8B-Base raises GSM8K strict EM from 79.4 to 94.2 and MathQA from 31.0 to 58.8.
• Echo reinsertion raises DeepSeek-R1-Distill-Llama-8B EM on failed GSM8K traces from 15.85% to 26.22%.
• Echoic Prompting consistently and substantially outperforms TTTS on AIME24 and MATH-500 with DeepSeek-R1-Distill-Llama-8B (exact deltas not given in text).

Surprising findings

• The suffix-only likelihood gap is larger for wrong traces (1.2938) than correct traces (1.1449), contrary to the main finding.
• Qwen3-8B-Base without CoT shows no gain from echo reinsertion (10.56% both branches).

Limitations

• Attention and likelihood analyses are performed on a single model and a single dataset (GSM8K with DeepSeek-R1-Distill-Llama-8B).
• Correlational evidence only for the echo-correctness link; the logistic regression coefficient is small and the causal claim rests on one reinsertion experiment.
• EP comparison against TTTS is reported only in figures without exact numbers in the text; ED-SFT trained only on GSM8K.

Experimental setup

Datasets: GSM8K MathQA Hendrycks-MATH AM-DeepSeek-R1-Distilled-1.4M

Benchmarks: GSM8K MathQA Hendrycks-MATH AIME24 MATH-500

Metrics: exact match accuracy Echo Likelihood Gap attention weight (answer to answer-prefix) Cohen's d AUC

Baselines: TTTS (thinking token test-time scaling) normal-SFT (CoT without echo) Qwen3-8B-Base Qwen3-8B DeepSeek-R1-Distill-Llama-8B

Builds on: chain-of-thought prompting thinking token based test-time scaling

Closest papers by meaning

• Stop When Further Reasoning Won’t Help: Attention-State Adaptive Generation in Reasoning Models (ICML 2026 · similarity 0.89)
• Bag of Tricks for Inference-time Computation of LLM Reasoning (NeurIPS 2025 · similarity 0.89)
• Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models (ICLR 2026 · similarity 0.89)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.88)
• Rethinking Thinking Tokens: LLMs as Improvement Operators (ICML 2026 · similarity 0.88)
• Fixing the Broken Compass: Diagnosing and Improving Inference-Time Reward Modeling (ICLR 2026 · similarity 0.88)