Research map / Reinforcement learning / Policy gradient and actor-critic
SPO: Sequential Monte Carlo Policy Optimisation
Matthew Macfarlane, Edan Toledo, Donal John Byrne, Paul Duckworth, Alexandre Laterre
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Sequential Monte Carlo Policy Optimisation (SPO), a self-play planning method that uses SMC-based search in place of tree search to scale planning during learning, as described in the abstract.
Problem
Tree-based search combined with self-play learning scales poorly because its search is sequential; engineering fixes that partly address this often sacrifice search quality or flexibility.
Approach
Replaces sequential tree search with a sequential Monte Carlo (SMC) planning procedure inside a policy optimisation loop, so that planning can be parallelised and used for learning. The provided text is only the NeurIPS template and the abstract's opening, so the specific mechanism is not confirmed from the text.
Experimental setup
Builds on: AlphaZero MCTS
Closest papers by meaning
- Efficient Multi-agent Reinforcement Learning by Planning (ICLR 2024 · similarity 0.88)
- Sequential Monte Carlo for Policy Optimization in Continuous POMDPs (NeurIPS 2025 · similarity 0.87)
- Exploring Starts Are Not Enough: Counterexamples and a Fix for Monte Carlo Exploring Starts (NeurIPS 2026 · similarity 0.87)
- Oracle-Efficient Reinforcement Learning for Max Value Ensembles (NeurIPS 2024 · similarity 0.87)
- Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Augmented Bayesian Policy Search (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
