Research map / Reinforcement learning / Visual and representation RL
The LoCA Regret: A Consistent Metric to Evaluate Model-Based Behavior in Reinforcement Learning
Harm Van Seijen, Hadi Nekoei, Evan Racah, Sarath Chandar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LoCA pretraining | MB-VI · Tabular gridworld | regret (x1000): 7.29 -> 0.00 | Improved |
| add LoCA pretraining | Sarsa(0.95) · Tabular gridworld | regret (x1000): 30.09 -> 14.92 | Improved |
| add LoCA pretraining | Q-learning · Tabular gridworld | regret (x1000): 68.25 -> 39.52 | Improved |
| use value iteration instead of single value update for planning (ablation) | MB-VI vs MB-SU · Tabular gridworld | LoCA regret (x1000): 1.72 -> 0.00 | Improved |
| step-size multiplier 0.1 (poor hyperparameters) (ablation) | MB-VI · Tabular gridworld | LoCA regret (x1000): 0.00 -> 0.00 | No clear effect |
| step-size multiplier 0.1 (poor hyperparameters) (ablation) | Sarsa(0.95) · Tabular gridworld | LoCA regret (x1000): 14.92 -> 134.82 | Hurt |
| expand state space by 5x with irrelevant features (ablation) | MB-VI · Tabular gridworld | LoCA regret (x1000): 0.00 -> 0.00 | No clear effect |
| expand state space by 2x with irrelevant features (ablation) | Sarsa(0.95) · Tabular gridworld | LoCA regret (x1000): 14.92 -> 26.30 | Hurt |
| expand state space by 10x with irrelevant features (ablation) | MB-SU · Tabular gridworld | LoCA regret (x1000): 1.72 -> 15.67 | Hurt |
| use 2-step on-policy model instead of 1-step (ablation) | tabular model-based method · Tabular gridworld | LoCA regret (x1000): 1.68 -> 24.87 | Hurt |
| use 5-step on-policy model instead of 1-step (ablation) | tabular model-based method · Tabular gridworld | LoCA regret (x1000): 1.68 -> 34.77 | Hurt |
| use 5-step on-policy model instead of 1-step (ablation) | tabular model-based method · Tabular gridworld | default regret (x1000): 42.98 -> 34.51 | Improved |
Closest papers by meaning
- Towards General-Purpose Model-Free Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning (ICLR 2024 · similarity 0.88)
- The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL (NeurIPS 2024 · similarity 0.87)
- A Theoretical Framework for Partially-Observed Reward States in RLHF (ICLR 2025 · similarity 0.87)
- Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
