Research map / Large language models / Reasoning and chain of thought

THE PATH OF LEAST RESISTANCE: GUIDING LLM REASONING TRAJECTORIES WITH PREFIX CONSENSUS

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PoLR, a training-free inference method that clusters short reasoning prefixes (TF-IDF + agglomerative clustering), expands only traces in the dominant cluster, and votes, cutting Self-Consistency token cost roughly in half while matching accuracy.

Problem

Self-Consistency decoding is expensive because every sampled reasoning trace must be fully expanded; adaptive variants still need complete traces before agreement is observed.

Approach

Exploit prefix consistency at inference time: cluster N short prefixes, identify the largest cluster, and fully expand only K traces from it before majority voting, generalizing SC (K=N without clustering reduces to SC).

Key results

• On GSM8K with QwQ-32B at N=51, PoLR matches SC accuracy (90.8%) with token efficiency η=47.6%.
• On AIME25, PoLR raises DSQ7B accuracy from 33.7% to 36.4% and Phi-4-15B from 32.0% to 36.0%, but drops QwQ-32B by 10 points (SC 76.7 vs PoLR 66.7 at N=51).
• PoLR+AC on GPQA-Diamond with DSQ7B (N=51) reduces path expansions from 18.05 (AC alone) to 10.58 at equal accuracy.

Surprising findings

• Cluster size correlates strongly with per-cluster accuracy (Pearson rho > 0.75), so the dominant cluster is the best choice to expand.
• Lightweight TF-IDF clustering matches dense Matryoshka sentence embeddings in accuracy and efficiency at a fraction of the overhead (~5-11 ms vs ~220 ms).

Limitations

• Accuracy drops on AIME25 for QwQ-32B (10 points, i.e. 3 of 30 problems) and on GPQA-Diamond for longer prefixes.
• Theoretical justification is a sketch; mutual information between cluster and correctness is weak (NMI <= 0.18), so the efficiency gain is attributed to skew rather than correctness alignment.
• Small evaluation sets (AIME 30 problems) make differences noisy.

Experimental setup

Datasets: GSM8K MATH500 AIME24 AIME25 GPQA-Diamond StrategyQA

Benchmarks: GSM8K MATH500 AIME24 AIME25 GPQA-Diamond StrategyQA

Metrics: accuracy Pass@1 exact match token efficiency path expansion clustering overhead (ms)

Baselines: Self-Consistency Chain-of-Thought (greedy) Adaptive Consistency Early-Stopping Self-Consistency PoLR with Matryoshka embeddings

Builds on: self-consistency adaptive consistency early-stopping self-consistency UPFT

Closest papers by meaning

• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Optimal Self-Consistency for Efficient Reasoning with Large Language Models (ICML 2026 · similarity 0.90)
• DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference (ICLR 2026 · similarity 0.90)
• R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing (NeurIPS 2025 · similarity 0.89)
• Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression (AAAI 2026 · similarity 0.89)
• The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models (NeurIPS 2025 · similarity 0.89)