Research map / Large language models / Reasoning and chain of thought

Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces twin tests (recoverability and guidability) for off-trajectory reasoning in solo-trained LLMs, evaluates 15 open-weight reasoning models, and runs controlled post-training studies on distillation teacher, RL, and data filtering.

Problem

Whether standard solo-reasoning training yields LLMs that can handle off-distribution thinking tokens from other reasoners on a shared trajectory, i.e., recovering from misleading steers and building on correct guidance from stronger collaborators.

Approach

Simulates two-model collaboration by splicing reasoning prefixes with steers: distractors sampled from the same model on a different question (recoverability), and correct partial reasoning from stronger guide models (guidability), then measures answer accuracy; isolates post-training factors with controlled distillation, GRPO, and data-selection experiments.

Key results

• AM-Thinking-32B, the top math model (82.6% benchmark average), recovers only 33.4% on the shared math subset, while Qwen3-1.7B (59.9%) recovers 98.4%.
• No model exceeds 9.2% guidability on the shared math subset, though about 18.6% of guiding steers already contain the correct answer.
• In controlled experiments, students distilled from AM-Thinking-32B show significantly lower recoverability than those from QwQ-32B or Qwen3-32B despite training only on correct teacher trajectories; GRPO after SFT largely closes the gap (recoverability gains of 15.3%-28.9%).

Surprising findings

• Stronger solo-reasoning models on benchmarks are often more fragile under distraction; benchmark rank is largely orthogonal to recoverability.
• Teacher recoverability weaknesses transfer to distilled students even when training uses only correct teacher trajectories.

Limitations

• Two-model setup with a single steer type, simplified relative to multi-agent collaboration.
• Guidability on math is confounded by correct answers already present in truncated steers.
• Evaluation restricted to correctness; safety-related collaboration not tested.

Experimental setup

Datasets: MATH8K LIMO AIME-2024 AIME-2025 MATH-500 Minerva Math OlympiadBench CruxEval HumanEval MBPP EvalPlus

Benchmarks: AIME-2024 AIME-2025 MATH-500 Minerva Math OlympiadBench CruxEval HumanEval MBPP EvalPlus

Metrics: Pass@1 recoverability rate guidability rate

Baselines: DeepSeek-R1 QwQ-32B Qwen3-32B AM-Thinking-32B LIMO-32B DeepScaleR-1.5B DeepMath-1.5B

Builds on: GRPO DeepSeek-R1 LIMO Math-Verify

Closest papers by meaning

• DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search (ICLR 2025 · similarity 0.89)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs (NeurIPS 2024 · similarity 0.88)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.88)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.88)
• How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning (ICML 2026 · similarity 0.88)
• Rethinking Thinking Tokens: LLMs as Improvement Operators (ICML 2026 · similarity 0.88)