AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

From POMDP Theory to Deep RL with Particle Filters

Kevin Tan, Ziping Xu

NeurIPS 2026 · Reinforcement learning › MDPs and planning · approach: POMDP policy optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Connects POMDP theory to practical deep RL by analyzing particle-filter-based belief inference, latent dynamics learning, and planning under belief in partially observable environments.

Problem

Model-based RL in partially observable environments requires inferring hidden states, learning latent dynamics, and planning under the belief; practical methods approximate these parts in poorly understood ways, while POMDP theory analyzes computationally inefficient algorithms.

Approach

Bridges POMDP theory and deep RL by using particle filters as the belief-inference component, with theoretical analysis of the full pipeline (inference, latent dynamics, planning) rather than one aspect in isolation. Details beyond the abstract are unknown.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase particle count J from 8 to 512 (ablation)PF-MCTQ · Mixing Tigerbelief-approximation error: 0.144 -> 0.012Improved
increase MCTS rollouts N from 4 to 512 (ablation)PF-MCTQ · Mixing TigerMonte Carlo RMSE: 0.171 -> 0.015Improved
increase planning depth d from 0 to 12 (reduce QMDP bias) (ablation)PF-MCTQ · Mixing Tigerstructural error: 0.314 -> 0.0216Improved
reduce policy update-to-data ratio to one update per ten steps, restore online model updates and weight decay (ablation)PF-MCTQ · MuJoCo Ant (partial observation)average return, 40k-49.2k steps: -396 -> 893 (position hidden); -518 -> 822 (velocity hidden)Improved
add GRU backbone instead of memoryless MLP (ablation)PF-MCTQ · Double-integrator LQRbelief trackingImproved
use PF-MCTQ instead of RLSVI-SMC (tailored linear-Gaussian method)RLSVI-SMC · HeartStepscumulative rewardHurt
use PF-MCTQ instead of recurrent model-free baselines in low-sample regimeRecurrent SAC / GRU-PPO · MuJoCo Hopper and Walker2d (partial observation)returnMixed
hide position instead of velocityPF-MCTQ · HalfCheetah (partial observation)returnHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy