Research map / Reinforcement learning / Offline RL
Offline Evaluation of Online Reinforcement Learning Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PSRS instead of a model-based (MLE MDP) evaluator | evaluation of PSRL · SixArms, 100 samples, 10 posterior samples | mean squared error of cumulative reward estimate | Helped |
| increase number of PSRL posterior samples (more revealed randomness) (ablation) | rejection sampling evaluators · Treefrog Treasure, horizon 3 | episodes evaluated per run | Helped |
| PERS instead of PSRS with randomized candidate policy | evaluator · Treefrog Treasure, horizon 4 | mean episodes evaluated: 173.52 -> 260.54 | Helped |
| PERS instead of PSRS under deterministic policies | evaluator · Treefrog Treasure, horizon 4 | episodes evaluated | No clear effect |
| hold M fixed in PERS (unbiased variant) (ablation) | PERS · Treefrog Treasure | episodes evaluated | Hurt |
| use PERS/Fixed-M PERS instead of Queue/PSRS on long horizon | evaluator · Riverswim, horizon 20 | episodes evaluated | Hurt |
| use Queue instead of PSRS | evaluator · Treefrog Treasure and Riverswim | episodes evaluated | Hurt |
Closest papers by meaning
- Simple Ingredients for Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- The Generalization Gap in Offline Reinforcement Learning (ICLR 2024 · similarity 0.90)
- Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design (ICML 2024 · similarity 0.90)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.89)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Scaling Offline Evaluation of Reinforcement Learning Agents through Abstraction (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.