Research map / Large language models / Reasoning and chain of thought
Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Dualformer trains a single Transformer on randomly trace-dropped A* search traces so it can run in fast (solution-only), slow (full trace), or auto mode on maze/Sokoban, and extends the idea to math LLM fine-tuning.
Problem
Transformers for planning either answer quickly without reasoning (fast mode, less accurate) or reason at length (slow mode, costly); prior systems need separate models or controllers to switch between these modes.
Approach
Train one encoder-decoder Transformer on A* search traces where structured parts of the trace (close clauses, cost tokens, create clauses, or the entire trace) are randomly dropped per example; a control token (plan or create) or no token selects fast, slow, or auto mode at inference.
Key results
- In slow mode on 30x30 mazes, Dualformer reaches 97.6% 1-Optimal-64 vs 93.3% for Complete-Trace (Searchformer) with about 45% shorter traces (854 vs 1538 tokens).
- In fast mode on 30x30 mazes, Dualformer reaches 80.0% 1-Optimal-64 vs 30.0% for Solution-Only.
- On MATH-derived AugMath, Mistral-7B slow-mode Greedy@1 improves from 16.9% to 18.6% with dropping probability p=0.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with randomized structured trace dropping rather than solution-only training | Dualformer fast mode · Maze 30x30 | 1-Optimal-64: 30.0 -> 80.0 | Helped |
| Train with randomized structured trace dropping rather than complete-trace training | Dualformer slow mode · Maze 30x30 | 1-Optimal-64: 93.3 -> 97.6 | Helped |
| Train with randomized structured trace dropping rather than complete-trace training | Dualformer slow mode · Maze 30x30 | Avg Trace Length: 1538 -> 854 | Helped |
| Use auto mode instead of Complete-Trace | Dualformer · Maze 30x30 | 1-Optimal-64: 93.3 -> 96.6 | Helped |
| Randomly drop reasoning steps during fine-tuning with p=0.1 | Mistral-7B · MATH (augmath) | Greedy@1 slow (%): 16.9 -> 18.6 | Helped |
| Randomly drop reasoning steps during fine-tuning with p=0.4 | Mistral-7B · MATH (augmath) | Pass@20 slow (%): 59.6 -> 56.4 | Hurt |
| Randomly drop reasoning steps during fine-tuning with p=0.2 | Llama-3-8B · MATH (augmath) | Greedy@1 slow (%): 19.7 -> 20.5 | Helped |
| Add Level 2 dropping to Level 1 dropping (ablation) | Dualformer · Maze 30x30 | 1-Optimal-64: 97.5 -> 98.1 | Helped |
| Add Level 3 dropping to Levels 1 and 2 (ablation) | Dualformer · Maze 30x30 | Avg Trace Length: 966 -> 759 | Helped |
| Add Level 3 dropping to Levels 1 and 2 (ablation) | Dualformer · Maze 30x30 | 1-Optimal-64: 98.1 -> 96.8 | Hurt |
| Add Level 4 dropping (drop the entire trace) to Levels 1, 2, and 3 (ablation) | Dualformer · Maze 30x30 | Avg Trace Length: 759 -> 854 | Mixed |
| Replace structured trace dropping with a 10% solution-only/complete-trace mixture (ablation) | Dualformer slow mode · Maze 30x30 | 1-Optimal-64: 97.7 -> 97.6 | Helped |
Surprising findings
- A model trained on 20x20 mazes generalizes to rectangular mazes of different shapes despite never seeing them in training.
- Dropping tokens reduced training time (30 hours vs 36 hours for full traces on 8 V100s).
Limitations
- Experiments on small models (15M and 46M parameters) trained from scratch on synthetic pathfinding tasks; scaling is untested.
- Performance drops for out-of-distribution wall densities (0.1, 0.2 and 0.6) when trained on 20x20 mazes.
- Math results show modest gains (e.g., about 1-2 points Greedy@1) over the baseline.
Experimental setup
Datasets: Maze and Sokoban A* search traces (Searchformer dataset) MATH AugMath (rewritten by Llama-3.1-70B-Instruct)
Benchmarks: 30x30 maze (and 15x15, 20x20, 25x25) Sokoban 7x7 MATH test set
Metrics: 1-Optimal-64 3-Optimal-64 1-Solved-64 3-Solved-64 SWC diversity (unique correct plans) average trace length Greedy@1 Pass@20
Baselines: Solution-Only Complete-Trace (Searchformer) Searchformer bootstrapped steps 1-3 Mix-p models Mistral-7B and Llama-3-8B fine-tuned on MATH-derived AugMath
Builds on: Searchformer dropout BERT masking T5
Closest papers by meaning
- Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.90)
- Selection, Reflection and Self-Refinement: Revisit Reasoning Tasks via a Causal Lens (ICLR 2026 · similarity 0.88)
- Rethinking Thinking Tokens: LLMs as Improvement Operators (ICML 2026 · similarity 0.88)
- SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction (ICLR 2025 · similarity 0.87)
- Controlling Thinking Speed in Reasoning Models (NeurIPS 2025 · similarity 0.87)
- Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.