Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Robust Active Measuring under Model Uncertainty
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces robust active-measuring MDPs (RAM-MDPs), a robust act-then-measure planning heuristic, and a measurement-lenient variant that bounds the return loss from extra measurements.
Problem
Sequential decision-making with both partial observability (costly measurements) and model uncertainty (imprecise transitions); robust POMDPs are hard to solve, and adversarial model uncertainty can make agents measure too little.
Approach
Extends MDPs to RAM-MDPs with interval transition uncertainty and costly complete noiseless measurements; adapts act-then-measure into a robust heuristic (RATM) that picks control actions from the underlying RMDP and measuring actions from a robust measuring value, plus measurement leniency that adds measurements under a less pessimistic model with a bounded return loss.
Key results
- On the drone environment (|S|=39,204), RATM-based methods outperform both ATM baselines in non-scalarized return, while ATM-pes slightly beats them on scalarized return.
- On snakemaze, the average-case ATM variant is outperformed by all other algorithms across confidence levels; measurement-lenient variants gain return under misspecified uncertainty for alpha > 0.7.
- Proves that measurement-lenient policies lose at most sum_n gamma^n c relative to their robust base policy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| account for how measurement affects the worst-case transition function | RATM · a-b | measurement behavior | Helped |
| use measurement leniency to encourage measurements under high uncertainty | R(C)ATM · lucky-unlucky | measurement behavior | Helped |
| use average- or optimistic-model measurement leniency | R(C)ATM · snakemaze, uncertainty α < 0.7 | scalarized return | Hurt |
| use average- or optimistic-model measurement leniency | R(C)ATM · snakemaze, misspecified uncertainty with α > 0.7 | scalarized return | Helped |
| use robust active-measure planning instead of ATMavg | R(C)ATM · drone | scalarized return | Helped |
| use robust active-measure planning instead of ATMavg or ATMpes | R(C)ATM · drone | non-scalarized return | Helped |
| add measurement leniency to robust active-measure planning | R(C)ATM · drone, misspecified uncertainty | non-scalarized return | No clear effect |
| use ATMavg instead of the other algorithms | ATM baseline · snakemaze | scalarized return | Hurt |
Surprising findings
- High model uncertainty can lead optimal robust policies to take fewer measurements, even when measuring would reduce the uncertainty.
- Measurement-lenient variants show inconsistent (stochastic) measuring behavior at certain costs due to ties in action values.
Limitations
- Computing the exact robust transition for RAM-MDPs is intractable, so experiments use the worst-case transition of the underlying RMDP.
- Measurements are restricted to complete and noiseless, and the heuristic is approximate.
- The per-step mixed integer program dominates online runtime.
Experimental setup
Datasets: snakemaze (custom gridworld) drone corridor (custom, 39,204 states) a-b environment lucky-unlucky environment
Benchmarks: a-b environment lucky-unlucky environment snakemaze drone corridor
Metrics: expected scalarized return non-scalarized return number of measurements runtime
Baselines: ATM (average-case environment) ATM (pessimistic RMDP environment) optimal policy (behavioral experiments)
Builds on: robust MDP active-measure MDP act-then-measure heuristic QMDP heuristic
Closest papers by meaning
- Efficient Value Iteration for s-rectangular Robust Markov Decision Processes (ICML 2024 · similarity 0.86)
- Model-Based Exploration in Monitored Markov Decision Processes (ICML 2025 · similarity 0.86)
- Sequential Monte Carlo for Policy Optimization in Continuous POMDPs (NeurIPS 2025 · similarity 0.86)
- Online MDP with Prototypes Information: A Robust Adaptive Approach (AAAI 2025 · similarity 0.86)
- Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy (NeurIPS 2024 · similarity 0.85)
- Revelations: A Decidable Class of POMDPs with Omega-Regular Objectives (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.