Research map / Architectures and efficiency / RNNs and normalization
One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models
Chris Cameron, Wangzheng Wang, Nikita Ivanov, Ashmita Bhattacharyya, Didier Chételat, Yingxue Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with denoising recursion (corrupt target, recover over k steps) instead of TRM backward-training | TRM 7M · ARC-Easy | pass@2 accuracy (%): 45.7 -> 50.5 | Improved |
| use DRM training instead of TRM | TRM 7M, no pretraining · ARC2-Eval | pass@2 accuracy (%): 6.3 -> 9.6 | Improved |
| use DRM training instead of TRM, NVARC pretraining | TRM 7M and 14M · ARC2-Eval (NVARC pretraining) | pass@2 accuracy (%): 12.5 -> 16.7 (7M); 21.8 -> 24.9 (14M) | Improved |
| add deep supervision (intermediate TBPTT losses) to TRM (ablation) | TRM 7M · ARC-Easy | pass@2 accuracy (%): 34.7 -> 45.7 | Improved |
| add recurrence (TRM) versus a standard non-recursive transformer (ablation) | transformer · ARC-Easy | pass@2 accuracy (%): 25.5 -> 45.7 | Improved |
| use standard one-step masked diffusion (no recursion backprop), even stacked to 70M (ablation) | masked diffusion · ARC-Easy | pass@2 accuracy (%): 0.0 (7M), 0.7 (70M) | Hurt |
| scale DRM from 7M/4 grad steps to 14M/6 grad steps (ablation) | DRM · ARC2-Train pretraining | pass@2 accuracy (%): +3-4% | Improved |
| add NVARC to pretraining data (ablation) | DRM 7M and 14M · ARC2-Eval | pass@2 accuracy (%): about +7% (7M), +11.5% (14M) | Improved |
| inject noise into latent states at TBPTT boundaries (SPRM) on TRM | TRM 7M · ARC2-Eval (no pretraining) | pass@2 accuracy (%): 6.3 -> 10.1 | Improved |
| SPRM state perturbation with reARC pretraining | TRM 7M · ARC2-Eval (reARC pretraining) | pass@2 accuracy (%): 12.4 -> 11.8 | Hurt |
| DRM (14M, NVARC) versus NVARC 4B LLM trained on same data | DRM 14M · ARC2-Eval (NVARC pretraining) | pass@2 accuracy (%): 22.6 -> 24.9 | Improved |
| use cosine instead of linear/sigmoid masking schedule (ablation) | DRM · ARC | training convergence | Improved |
Closest papers by meaning
- Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning? (ICML 2024 · similarity 0.87)
- Looped Diffusion Language Models (NeurIPS 2026 · similarity 0.87)
- Encoder-Decoder Diffusion Language Models for Efficient Training and Inference (NeurIPS 2025 · similarity 0.87)
- Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism (NeurIPS 2025 · similarity 0.87)
- Denoising Task Difficulty-based Curriculum for Training Diffusion Models (ICLR 2025 · similarity 0.87)
- Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
