Research map / Reinforcement learning / Policy gradient and actor-critic
Robust Losses for Learning Value Functions
Andrew P. Patterson, Victor Liao, Martha White
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Huber Bellman error instead of squared Bellman error (MHBE vs MSBE fixed point) | linear value function · HardAlias-2 | value error | Improved |
| use robust objectives instead of MSBE on idealized chains | linear value function · SmallChain/BigChain | value error | No clear effect |
| use L1 gradient-correction (L1-TDC) instead of GTD2 variant | gradient TD · linear prediction | MSVE | Hurt |
| use robust TDC variants instead of squared TDC | gradient TD · Baird counterexample | MSVE | Improved |
| use QRC-Huber instead of QRC (squared loss) | QRC · CliffWorld | average return | Improved |
| use QRC-Huber instead of DQN | DQN · classic control (5 domains) | average return | Improved |
| add target networks to QRC and QRC-Huber (ablation) | QRC-Huber · classic control | average return | Hurt |
| remove target networks from DQN (ablation) | DQN · Cart-pole | average return | Hurt |
| use QRC-Huber instead of QRC | QRC · Minatar | scaled return | Improved |
| use QRC-Huber instead of DQN | DQN · Minatar | scaled return | Improved |
| use saddlepoint GQ instead of gradient-correction QRC-Huber (ablation) | nonlinear control · classic control | mean return | Hurt |
| use smaller Huber threshold tau (ablation) | QRC-Huber · Mountain Car | performance distribution | Hurt |
Closest papers by meaning
- A Reduction Framework for Distributionally Robust Reinforcement Learning under Average Reward (ICML 2025 · similarity 0.88)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.88)
- Calibrated Value-Aware Model Learning with Probabilistic Environment Models (ICML 2025 · similarity 0.87)
- Robust Offline Reinforcement Learning with Linearly Structured $f$-Divergence Regularization (ICML 2025 · similarity 0.87)
- Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning (ICLR 2024 · similarity 0.87)
- Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
