AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

SPO: Sequential Monte Carlo Policy Optimisation

Matthew Macfarlane, Edan Toledo, Donal John Byrne, Paul Duckworth, Alexandre Laterre

NeurIPS 2024 · Reinforcement learning › Policy gradient and actor-critic · approach: Tree search and policy optimization · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Sequential Monte Carlo Policy Optimisation (SPO), a self-play planning method that uses SMC-based search in place of tree search to scale planning during learning, as described in the abstract.

Problem

Tree-based search combined with self-play learning scales poorly because its search is sequential; engineering fixes that partly address this often sacrifice search quality or flexibility.

Approach

Replaces sequential tree search with a sequential Monte Carlo (SMC) planning procedure inside a policy optimisation loop, so that planning can be parallelised and used for learning. The provided text is only the NeurIPS template and the abstract's opening, so the specific mechanism is not confirmed from the text.

Experimental setup

Builds on: AlphaZero MCTS

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy