Research map / Reinforcement learning / Policy gradient and actor-critic
Controlling Underestimation Bias in Reinforcement Learning via Quasi-median Operation
Wei Wei, Yujia Zhang, Jiye Liang, Lin Li, Yyuze Li
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use quasi-median aggregation of multiple critics | QMD3 · HalfCheetah-v3 | average return: 9465.3 -> 12221.9 | Improved |
| Use quasi-median aggregation of multiple critics | QMD3 · Walker2d-v3 | average return: 4449.3 -> 4765.3 | Improved |
| Use quasi-median aggregation of multiple critics | QMD3 · Humanoid-v3 | average return: 4719.8 -> 5332.4 | Improved |
| Use quasi-median aggregation of multiple critics | QMD3 · IDP-v2 | average return: 9237.4 -> 9891.3 | Improved |
| Add the quasi-median operation (ablation) | QMD3 on HalfCheetah · HalfCheetah | average return: 10576.3 -> 11763.5 | Improved |
| Add the quasi-median operation (ablation) | QMD3 on Hopper · Hopper | average return: 3248.9 -> 3663.4 | Improved |
| Add the quasi-median operation (ablation) | QMD3 on Walker2d · Walker2d | average return: 4103.6 -> 4424.5 | Improved |
| Add the quasi-median operation (ablation) | QMD3 on Ant · Ant | average return: 4286.3 -> 4786.4 | Improved |
| Add Exploration Improvement by updating the actor using multiple critics (ablation) | QMD3 on HalfCheetah · HalfCheetah | average return: 11276.0 -> 11763.5 | Improved |
| Add Exploration Improvement by updating the actor using multiple critics (ablation) | QMD3 on Hopper · Hopper | average return: 3472.8 -> 3663.4 | Improved |
| Add Exploration Improvement by updating the actor using multiple critics (ablation) | QMD3 on Walker2d · Walker2d | average return: 4354.4 -> 4424.5 | Improved |
| Add Exploration Improvement by updating the actor using multiple critics (ablation) | QMD3 on Ant · Ant | average return: 4534.2 -> 4786.4 | Improved |
Closest papers by meaning
- Gaussian Approximation for Bias Reduction in Q-Learning (JMLR 2021 · similarity 0.90)
- Exclusively Penalized Q-learning for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.87)
- Q-Distribution guided Q-learning for offline reinforcement learning: Uncertainty penalized Q-value via consistency model (NeurIPS 2024 · similarity 0.87)
- Learning from Suboptimal Data in Continuous Control via Auto-Regressive Soft Q-Network (ICML 2025 · similarity 0.87)
- Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
