Research map / Large language models / Reasoning and chain of thought

Selection, Reflection and Self-Refinement: Revisit Reasoning Tasks via a Causal Lens

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Casts reasoning as a causal selection mechanism and proposes SR², a flattened recurrent single-Transformer-layer model with reflective input injection, dependency self-refinement, and periodic intermediate alignment, reaching state-of-the-art on Sudoku-Extreme and Maze-Hard with 3.4M parameters.

Problem

Pure LLMs and standard deep networks fail to perform reliable multi-step reasoning; the paper seeks to explain why reasoning is hard via latent-space complexity and dense dependencies among latent logical rules, and to design architectures that model them.

Approach

Reframes reasoning as a selection mechanism over latent rules, then implements it as a single shared Transformer block iterated with input re-injection (reflection) followed by input-free self-refinement, with supervision injected periodically at intervals and gradients detached per block.

Key results

• SR² reaches 66.63% on Sudoku-Extreme and 93.7% on Maze-Hard with 3.4M parameters, versus HRM's 55.0% and 74.5% at 27.3M.
• On ARC-AGI-1 and ARC-AGI-2, SR² scores 44.3% and 6.7% versus HRM's 40.3% and 5.0%.
• Ablation removing reflection (no input injection) collapses accuracy to 0% on Sudoku-Extreme, while removing self-refinement drops it to 53.11%.

Surprising findings

• Standard Transformer scores 0% on Maze-Hard and 0% on ARC-AGI-2, and Block Universal Transformer scores 0% on Sudoku-Extreme despite comparable parameter budgets.
• On a CUB-200-2011 image classification test, SR² reaches under 45% top-1 accuracy versus roughly 75% for a ViT baseline, suggesting iterative latent refinement is not universally beneficial.

Limitations

• The selection-mechanism assumption may oversimplify human reasoning.
• Evaluation is limited to benchmarks with clear, simple logical constraints; no evaluation on complex or open-domain tasks or large-scale LLM settings.
• Performance on image classification (CUB) is much worse than a ViT baseline, indicating limited applicability.

Experimental setup

Datasets: Sudoku-Extreme Maze-Hard ARC AGI 1 ARC AGI 2 CUB-200-2011

Benchmarks: Sudoku-Extreme Maze-Hard ARC AGI 1 ARC AGI 2

Metrics: accuracy (pass@1) pass@2 parameter count training speed inference speed

Baselines: Standard Transformer Block Universal Transformer Recurrent Depth HRM Reflective Model

Builds on: Hierarchical Reasoning Model Deep Equilibrium Models Universal Transformer Recurrent Depth

Closest papers by meaning

• Efficient Reasoning via Constrained Optimization in Latent Space (NeurIPS 2026 · similarity 0.89)
• Efficient Post-Training Refinement of Latent Reasoning in Large Language Models (AAAI 2026 · similarity 0.89)
• Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.89)
• Decoding the Critique Mechanism in Large Reasoning Models (NeurIPS 2026 · similarity 0.89)
• RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation (ICML 2025 · similarity 0.89)
• Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement (ICLR 2024 · similarity 0.89)