Research map / Reinforcement learning / Policy gradient and actor-critic
Minimax Value Interval for Off-Policy Evaluation and Policy Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the unified minimax value interval instead of the naive MQL-derived interval | Off-policy evaluation on CartPole · CartPole | interval length | Helped |
| Bootstrap the value interval to handle sampling errors | Off-policy evaluation on CartPole · CartPole | validity: 100% | Helped |
| Increase Q-network size (ablation) | Minimax value interval · CartPole | bound ordering | Mixed |
| Use a small, poorly specified Q-function class (ablation) | Minimax value interval · CartPole | bound ordering | Mixed |
| Add nontrivial regularization (ablation) | Minimax value interval · Off-policy evaluation | interval tightness | Hurt |
| Optimize the lower bound rather than the upper bound | Off-policy policy optimization with insufficient data coverage · Tabular MDP with uncovered states | worst-case return guarantee | Helped |
| Optimize the upper bound rather than the lower bound | Off-policy policy optimization with insufficient data coverage · Tabular MDP with uncovered states | exploration | Helped |
| Use MLB-PO instead of DQN with non-exploratory behavior data | Off-policy policy optimization · Near-deterministic behavior policy | policy performance stability | Helped |
| Use DQN instead of MLB-PO with non-exploratory behavior data | Off-policy policy optimization · Near-deterministic behavior policy | performance variance | Hurt |
| Use MUB-PO for exploitation | Off-policy policy optimization · Near-deterministic behavior policy | policy value | Hurt |
Closest papers by meaning
- Beyond the Return: Off-policy Function Estimation under User-specified Error-measuring Distributions (NeurIPS 2022 · similarity 0.91)
- From Importance Sampling to Doubly Robust Policy Gradient (ICML 2020 · similarity 0.91)
- A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes (ICML 2022 · similarity 0.91)
- Minimax Weight and Q-Function Learning for Off-Policy Evaluation (ICML 2020 · similarity 0.91)
- Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.88)
- Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.