Research map / Reinforcement learning / RL from feedback
Latent Reasoning in TRMs is Secretly a Policy Improvement Operator
Arip Asadulaev, Rayan Banerjee, Fakhri Karray, Martin Takáč
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Deep Improvement Supervision (stepwise intermediate targets) to TRM, compact model | TRM-compact 0.8M · ARC-AGI-1 (pass@2) | accuracy: 12.0 -> 24.0 | Improved |
| add DIS to medium TRM with same reduced recursion (T=1, n=2) | TRM-medium 7M · ARC-AGI-1 (pass@2) | accuracy: 27.1 -> 40.0 | Improved |
| add DIS to medium TRM | TRM-medium 7M · ARC-AGI-2 (pass@2) | accuracy: 0.0 -> 2.5 | Improved |
| replace TRM with DIS at full size | TRM 7M · ARC-AGI-2 (pass@2) | accuracy: 3.3 -> 6.0 | Improved |
| replace TRM with DIS at full size | TRM 7M · ARC-AGI-1 (pass@2) | accuracy: 40.4 -> 41.3 | Improved |
| reduce recursion from T=3,n=6 (336 steps) to T=1,n=2 (18 steps) with DIS, remove halting head | TRM on N-Queens · N-Queens 8x8 | accuracy: 0.69 -> 0.69 | No clear effect |
| add DIS on compact model for ARC-AGI-2 | TRM-compact 0.8M · ARC-AGI-2 (pass@2) | accuracy: 0.0 -> 0.0 | No clear effect |
| use integer time-step index instead of continuous diffusion time conditioning (ablation) | DIS · ARC/N-Queens | accuracy | Improved |
| DIS training with fixed 6 supervision steps, no halting head | TRM training · training time on 4 H100 | training time: 72 -> about 40 hours | Improved |
Closest papers by meaning
- Hybrid Latent Reasoning via Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models (NeurIPS 2026 · similarity 0.88)
- Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.88)
- Efficient Reasoning via Constrained Optimization in Latent Space (NeurIPS 2026 · similarity 0.87)
- Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models (ICML 2026 · similarity 0.87)
- Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
