Research map / Reinforcement learning / Policy gradient and actor-critic
Combining parametric and nonparametric models for off-policy evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine parametric and nonparametric models, switching to the data-based model where observed transitions are abundant | MoE · 2D motivating example | value estimate accuracy: true -40; Parametric -18; Nonparametric -$\infty$; MoE -31 | Helped |
| Plan model choices over entire trajectories instead of greedily minimizing immediate transition error | MoE · 2D planning toy, accurate reward model | value estimation error: 39.5 -> 17 | Helped |
| Plan model choices over entire trajectories instead of greedily minimizing immediate transition error | MoE · 2D planning toy, inaccurate reward model | value estimation error: 41.5 -> 19.5 | Helped |
| Combine parametric and nonparametric models using greedy model selection | Model-based OPE · Cancer | relative value estimation RMSE: 0.021 -> 0.020 | Helped |
| Add planning over model choices | MoE · Cancer | relative value estimation RMSE: 0.020 -> 0.019 | Helped |
| Combine parametric and nonparametric models using greedy model selection | Model-based OPE · HIV | relative value estimation RMSE: 0.65 -> 0.64 | Helped |
| Add planning over model choices | MoE · HIV | relative value estimation RMSE: 0.64 -> 0.63 | Helped |
| Use the nonparametric model alone rather than mixing models | MoE · Acrobot | value estimate RMSE | Mixed |
| Use a model-based estimator rather than importance-sampling estimators | Off-policy evaluation · Cancer and HIV | value estimation error | Helped |
| Use Euclidean distance for state error when one state dimension matters more to reward | MCTS-MoE · HIV | policy evaluation error | Hurt |
Closest papers by meaning
- Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.91)
- Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling (NeurIPS 2019 · similarity 0.91)
- More Efficient Off-Policy Evaluation through Regularized Targeted Learning (ICML 2019 · similarity 0.91)
- Off-Policy Evaluation via Off-Policy Classification (NeurIPS 2019 · similarity 0.91)
- COMBO: Conservative Offline Model-Based Policy Optimization (NeurIPS 2021 · similarity 0.90)
- Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.