Research map / Bandits and online learning / RL theory and MDPs
Model-Based Reinforcement Learning with Value-Targeted Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use value-targeted regression with optimistic planning instead of next-state frequency estimation with optimistic planning | UCRL-VTR vs. UC-MatrixRL · RiverSwim | cumulative regret | Helped |
| Add optimistic planning to value-targeted regression rather than using epsilon-greedy exploration (ablation) | Value-targeted regression · RiverSwim | cumulative regret | Helped |
| Add optimistic planning to value-targeted regression rather than using epsilon-greedy exploration (ablation) | Value-targeted regression · WideTree | cumulative regret | Helped |
| Replace value-targeted regression with next-state frequency estimation while retaining optimism (ablation) | Optimistic model-based RL · RiverSwim | cumulative regret | Hurt |
| Remove optimistic planning from value-targeted regression and use epsilon-greedy exploration (ablation) | UCRL-VTR · WideTree | cumulative regret | Hurt |
| Replace value-targeted regression with next-state frequency estimation while using epsilon-greedy exploration (ablation) | Epsilon-greedy model-based RL · RiverSwim | weighted model error | Mixed |
| Use value-targeted regression without optimistic planning (ablation) | EG-VTR · WideTree | weighted model error | No clear effect |
| Use value-targeted regression rather than next-state frequency estimation (ablation) | Epsilon-greedy model-based RL · WideTree | cumulative regret | No clear effect |
| Use value-targeted regression with optimistic planning, even if the learned transition model remains inaccurate | UCRL-VTR · WideTree | cumulative regret | Helped |
Closest papers by meaning
- Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Bounds (ICLR 2025 · similarity 0.91)
- Tight Regret Bounds for Model-Based Reinforcement Learning with Greedy Policies (NeurIPS 2019 · similarity 0.91)
- Horizon-Free Regret for Linear Markov Decision Processes (ICLR 2024 · similarity 0.90)
- Achieving Constant Regret in Linear Markov Decision Processes (NeurIPS 2024 · similarity 0.90)
- Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning (ICLR 2024 · similarity 0.89)
- Minimax Optimal Reinforcement Learning with Quasi-Optimism (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.