AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

Belief-Dependent Macro-Action Discovery in POMDPs using the Value of Information

Genevieve Flaspohler, Nicholas A. Roy, John W. Fisher III

NeurIPS 2020 · Reinforcement learning › MDPs and planning · approach: POMDP policy optimization · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use VoI-based macro-actions instead of closed-loop planningPOMDP planner · Boundary Dynamics, alpha=0.20total reward: 2800.2 -> 3039.5Improved
use VoI-based macro-actions instead of closed-loop planningPOMDP planner · Random Walk Dynamicstotal reward: 3035.8 -> 3050.8No clear effect
use VoI macro-actions instead of fixed-length macro-actions (T=15)POMDP planner · Boundary Dynamics, alpha=0.20total reward: 2584.3 -> 3039.5Improved
use VoI macro-actions instead of fixed-length macro-actions (T=15)POMDP planner · Random Walk Dynamicstotal reward: 2946.6 -> 3050.8Improved
use VoI macro-actions instead of closed-loop planningPOMDP planner · Boundary Dynamics, alpha=0.20empirical delta_B: 0.45 -> 0.29Improved
use VoI macro-actions instead of closed-loop planningPOMDP planner · Random Walk Dynamicsempirical delta_B: 0.42 -> 0.31Improved
use fixed-length macro-actions instead of VoI macro-actionsPOMDP planner · Boundary Dynamics, alpha=0.20empirical delta_B: 0.29 -> 0.17Improved
use VoI macro-actions instead of closed-loop planningPOMDP planner · Experiment 2 (zone of interest)average reward: 15.08 -> 18.69Improved
use VoI macro-actions instead of fixed-length macro-actionsPOMDP planner · Experiment 2 (zone of interest)average reward: 13.03 -> 18.69Improved
increase VoI threshold tau (ablation)VoI macro-action policy · Boundary trackingempirical delta_BImproved
increase VoI threshold tau (ablation)VoI macro-action policy · Boundary trackingopen-loop fraction of horizonImproved
increase VoI threshold tau (ablation)VoI macro-action policy · Boundary trackingrealized rewardMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy