Research map / Learning theory and algorithms / Combinatorial and search optimization

Simplifying Complex Observation Models in Continuous POMDP Planning with Probabilistic Guarantees and Practice

AAAI 2024 · Learning theory and algorithms › Combinatorial and search optimization · approach: Classical and HTN planning · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives probabilistic (concentration) bounds on the value loss from planning in continuous POMDPs with a simplified observation model, using a state-dependent TV-distance estimated offline and reweighted online, and integrates it into PFT-DPW on a 2D light-dark beacon task.

Problem

Planning in continuous POMDPs with high-dimensional observations requires expensive learned observation models; it is unclear how much a simplified observation model degrades the planned value, and existing continuous solvers lack such guarantees.

Approach

A state-dependent total variation distance between original and simplified observation models is bounded in value via a cumulative bound M and action-cumulative bound Phi, estimated online by importance-reweighting offline pre-sampled delta states, so the costly original model is never accessed during planning; concentration results extend particle-belief MDP bounds to general policies and state r

Key results

• Simplified observation model planning with PFT-DPW (500 simulations per step, 100 scenarios) has greatly reduced planning time compared with the original GMM observation model.
• Lower-bound and upper-bound policies differ substantially from the simplified value policy, especially in time steps 1-7 when the agent is near the light region.
• The bound is shown to be probabilistically valid: Q-values of the original POMDP are bounded with high probability by simplified PB-MDP value plus the estimated cumulative bound Phi.

Measured effects reported in the paper

Surprising findings

• The bound is independent of the chosen policy, so the bound-based action choice can be computed from simplified-model quantities alone.

Limitations

• Experiments are limited to a simple 2D simulated light-dark environment with a single Gaussian approximation of a GMM.
• Requires assumptions such as bounded likelihood ratio (d_inf max) and bounded support of the state space for the estimator to be consistent.
• Concentration bounds are loose and no performance guarantees are demonstrated on realistic visual observation models.

Experimental setup

Datasets: 2D beacons light-dark simulated POMDP

Benchmarks: 2D light-dark beacons simulation

Metrics: planning duration fraction of actions differing from simplified value policy estimated value and action cumulative bound

Baselines: PFT-DPW with original observation model lower bound policy upper bound policy

Builds on: PFT-DPW particle-belief MDP concentration bounds POWSS Sparse Sampling

Closest papers by meaning

• Sequential Monte Carlo for Policy Optimization in Continuous POMDPs (NeurIPS 2025 · similarity 0.89)
• PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty (NeurIPS 2026 · similarity 0.89)
• Factored Online Planning in Many-Agent POMDPs (AAAI 2024 · similarity 0.88)
• Provable Representation with Efficient Planning for Partially Observable Reinforcement Learning (ICML 2024 · similarity 0.87)
• Multi-Environment POMDPs: Discrete Model Uncertainty Under Partial Observability (NeurIPS 2025 · similarity 0.87)
• Risk-Averse Online POMDP Planning via CVaR of the Immediate Cost with Performance Guarantees (NeurIPS 2026 · similarity 0.87)