Research map / Large language models / Reasoning and chain of thought

Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains small Qwen2.5-0.5B-architecture transformers from scratch on maze pathfinding with formally validated A* traces, showing trace validity is weakly tied to solution correctness and that swapped (irrelevant) traces perform comparably or better, especially OOD.

Problem

Whether the intermediate reasoning tokens (chains of thought) of language models carry end-user semantics that causally drive solution accuracy, and whether trace length reflects problem complexity.

Approach

Controlled model-organism study: train from-scratch transformers on A* search traces that can be formally validated, then compare against "swapped" datasets where traces are randomly permuted across problems, and test GRPO and length scaling effects on trace validity.

Key results

• Swapped-trace Wilson model reaches 11.7% plan accuracy on Drunkard vs 0.0% for the normal model; swapped SF-style model reaches 95.4% on Drunkard vs 62.1% for normal.
• Normal SF-style model's trace validity among valid plans falls from ~85% to ~50% after GRPO while plan accuracy rises (e.g., Wilson test 40.8% to 63.7%).
• Trace length correlates weakly with A* trace length for SF-style model (Pearson r=0.197) and shows no correlation for OOD SF-style mazes (r=0.02).

Surprising findings

• Models trained on swapped (0% valid) traces match or exceed models trained on correct traces, and generalize better OOD.
• GRPO post-training can reduce trace validity while increasing plan accuracy.

Limitations

• Experiments are limited to small from-scratch models on synthetic grid maze pathfinding, limiting generality to frontier LLMs.
• Pre-trained Qwen3-8B experiments are preliminary and confounded by unknown pretraining data.
• Trace validity only checks A* rule compliance and does not check optimality.

Experimental setup

Datasets: Wilson maze dataset (A* traces) Searchformer-style maze dataset Swapped trace datasets Kruskal, DFS, Drunkard test mazes

Benchmarks: Wilson maze test set Kruskal maze test set DFS maze test set Drunkard maze test set SF-style maze test set

Metrics: plan accuracy trace validity within valid plans trace length correlation (Pearson, Spearman)

Baselines: Solution-only baseline Searchformer

Builds on: Searchformer Stream of Search GRPO Qwen2.5

Closest papers by meaning

• Measuring Weak-to-Strong Legibility of Reasoning Models (NeurIPS 2026 · similarity 0.91)
• Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Trace (NeurIPS 2026 · similarity 0.91)
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (NeurIPS 2025 · similarity 0.91)
• On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study (NeurIPS 2025 · similarity 0.90)
• Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.90)
• Probing the Trajectories of Reasoning Traces in Large Language Models (NeurIPS 2026 · similarity 0.90)