Research map / Probabilistic and scientific modeling / Molecules and proteins
Monte Carlo Tree Diffusion for System 2 Planning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Monte Carlo Tree Diffusion (MCTD), which embeds MCTS steps into diffusion planning by treating partially denoised subplans as tree nodes, using guidance levels as meta-actions and jumpy DDIM denoising as simulation, and reports near-perfect OGBench maze success.
Problem
Diffusion-based planners offer limited inference-time scalability, while MCTS scales with compute but relies on forward models and discrete, small action spaces; the paper asks how to combine them for long-horizon planning.
Approach
Reconceptualize denoising as a tree rollout over semi-autoregressive subplans, where each node is a partially denoised subplan, edges are binary guidance meta-actions (guide / no-guide) chosen via UCB, and value is estimated by fast jumpy DDIM denoising of the remainder followed by reward evaluation.
Key results
- On pointmaze giant-navigate, MCTD reaches 100% success versus 0% for Diffuser and 50% for Diffusion Forcing.
- On antmaze giant-navigate, MCTD reaches 94±9% versus 0% for Diffuser and 24% for Diffusion Forcing.
- On double-play cube manipulation, MCTD-Replanning improves success from 22% (MCTD) to 50%, and to 78% with a single-cube-trained policy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace one-shot diffusion planning with MCTD tree search | Diffusion planning · pointmaze giant-navigate-v0 | success rate (%): 0 ± 0 -> 100 ± 0 | Helped |
| Replace Diffusion Forcing with MCTD | Long-horizon maze planning · antmaze giant-navigate-v0 | success rate (%): 24 ± 12 -> 94 ± 9 | Helped |
| Periodically replan using MCTD | MCTD cube manipulation · double-play-v0 | success rate (%): 22% to 50% | Helped |
| Use MCTD-Replanning instead of MCTD | Visual pointmaze planning · visual pointmaze large-navigate-v0 | success rate (%): 0±0 -> 20 ± 21 | Helped |
| Use greedy candidate selection instead of statistical tree search (ablation) | MCTD · PointMaze giant | success rate (%): 100±0 -> 0±0 | Hurt |
| Use an exploration-biased guidance set (Set 1) instead of the default balanced set (Set 2) (ablation) | MCTD · PointMaze giant | success rate (%): 100±0 -> 54±23 | Hurt |
| Use an exploitation-biased guidance set (Set 4) instead of the default balanced set (Set 2) (ablation) | MCTD · PointMaze giant | success rate (%): 100±0 -> 74±18 | Hurt |
| Set subplan count to 5 rather than 1 (ablation) | MCTD · PointMaze large | success rate (%): 80±0 -> 100±0 | Helped |
| Set subplan count to 20 rather than 5 (ablation) | MCTD · PointMaze large | success rate (%): 100±0 -> 92±10 | Hurt |
| Add causal denoising (ablation) | MCTD without causal denoising · PointMaze medium | success rate (%): 87±16 -> 100±0 | Helped |
| Add causal denoising (ablation) | MCTD without causal denoising · PointMaze giant | success rate (%): 18±11 -> 100±0 | Helped |
Surprising findings
- Diffuser-Random Search, using roughly the same denoising budget as MCTD, shows no improvement over one-shot Diffuser.
- A greedy tree search baseline performs poorly (e.g., 0% on giant pointmaze), indicating that backtracking and deeper search matter.
Limitations
- Computationally expensive compared with single-pass diffusion planning due to tree search overhead.
- Scalability limited in large search spaces; reliance on hand-designed reward and low-level controllers (heuristic controller, value-learning policy).
- Visual pointmaze large-maze performance remains low (20% for MCTD-Replanning).
Experimental setup
Datasets: OGBench pointmaze navigate OGBench antmaze navigate OGBench cube-play single/double/triple/quadruple visual pointmaze (generated from OGBench script) pointmaze stitch
Benchmarks: OGBench pointmaze OGBench antmaze OGBench cube manipulation visual pointmaze
Metrics: success rate (%) wall-clock runtime search count
Baselines: Diffuser Diffuser-Replanning Diffuser-Random Search Diffusion Forcing Monte Carlo Guidance Goal-Inpainting Diffuser
Builds on: Diffuser Diffusion Forcing Monte Carlo Tree Search DDIM PlanDQ
Closest papers by meaning
- Fast Monte Carlo Tree Diffusion: 100× Speedup via Parallel and Sparse Planning (NeurIPS 2025 · similarity 0.94)
- Compositional Monte Carlo Tree Diffusion for Extendable Planning (NeurIPS 2025 · similarity 0.93)
- Resisting Stochastic Risks in Diffusion Planners with the Trajectory Aggregation Tree (ICML 2024 · similarity 0.89)
- DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising (NeurIPS 2026 · similarity 0.88)
- Diffusion Tree Sampling: Scalable inference‑time alignment of diffusion models (NeurIPS 2025 · similarity 0.87)
- Simple Hierarchical Planning with Diffusion (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.