Research map / Reinforcement learning / MDPs and planning
Target Surveillance in Adversarial Environments Using POMDPs
Maxim Egorov, Mykel Kochenderfer, Jaak Uudmae
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SARSOP instead of QMDP to solve the MOMDP | level 3 surveillance policy · Map B vs level 2 SARSOP threat | average reward: 0.32 ± 0.4 -> 0.96 ± 0.4 | Improved |
| use SARSOP MOMDP policy instead of greedy heuristic | level 3 surveillance policy · Map B vs level 2 SARSOP threat | average reward: -0.21 ± 0.1 -> 0.96 ± 0.4 | Improved |
| SARSOP policy vs fully observable nested MDP upper bound | level 3 surveillance policy · Map B vs level 2 SARSOP threat | average reward: 2.30 ± 0.4 -> 0.96 ± 0.4 | Hurt |
| adversary follows intelligent SARSOP policy instead of random | SARSOP surveillance policy · Map B | average reward: 4.23 ± 0.4 -> 0.96 ± 0.4 | Hurt |
| increase reasoning level k of surveillance agent to 3 (ablation) | level-k SARSOP surveillance policy · level-k threats, Figure 5 | average reward | Improved |
| level 3 policy vs level 1 policy (modelling adversary as level-k vs naive) (ablation) | SARSOP surveillance policy · video game vs level 2 SARSOP adversary | survival vs observation rate curve | Improved |
| SARSOP policy vs human players | surveillance agent · video game vs level 2 SARSOP adversary | survival probability | Improved |
Closest papers by meaning
- Optimal Attack and Defense for Reinforcement Learning (AAAI 2024 · similarity 0.83)
- Multi-Environment POMDPs: Discrete Model Uncertainty Under Partial Observability (NeurIPS 2025 · similarity 0.82)
- Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL (ICLR 2024 · similarity 0.82)
- Solving Robust Markov Decision Processes: Generic, Reliable, Efficient (AAAI 2025 · similarity 0.82)
- Multi-Environment POMDPs with Finite-Horizon Objectives (NeurIPS 2026 · similarity 0.82)
- Sequential Monte Carlo for Policy Optimization in Continuous POMDPs (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
