Research map / Reinforcement learning / Policy gradient and actor-critic
A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning
Andrew Patterson, Adam White, Martha White
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use emphatic weighting m instead of behavior distribution db in the PBE (ablation) | PBE fixed point · 19-state random walk (Figure 5) | value error of fixed point | Improved |
| use PBE instead of BE (restricted H instead of full projection) (ablation) | value estimation fixed point · 19-state random walk (Figure 5) | value error of fixed point | No clear effect |
| use target-policy weighting d_pi instead of emphatic weighting m for PBE (ablation) | PBE fixed point · 19-state random walk, VE under d_pi | value error under d_pi | Mixed |
Closest papers by meaning
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- Neural Network Approximation for Pessimistic Offline Reinforcement Learning (AAAI 2024 · similarity 0.90)
- Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning (ICLR 2024 · similarity 0.88)
- Parameterized Projected Bellman Operator (AAAI 2024 · similarity 0.88)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
