Research map / Reinforcement learning / Policy gradient and actor-critic
Gaussian Approximation for Bias Reduction in Q-Learning
Carlo D'Eramo, Andrea Cini, Alessandro Nuara, Matteo Pirotta, Cesare Alippi, Jan Peters, Marcello Restelli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the Maximum Estimator with the Weighted Estimator in absolute-bias comparison of two Gaussian variables | Weighted Estimator · Two Gaussians, sigma=10, 100 samples each | absolute bias: between 0.5 and 1.8 | Mixed |
| use Weighted Estimator instead of Double Estimator when means differ by little | Weighted Estimator · Two Gaussians, sigma=10, 100 samples each | absolute bias: less than 0.5 | Hurt |
| use WE instead of ME, DE, MME in internet ads MAB | Weighted Estimator · Internet ads (CTR estimation) | MSE | Improved |
| use WE instead of ME in sponsored search auctions | Weighted Estimator · Sponsored search auctions | incentive compatibility | Improved |
| use Double Estimator instead of ME in sponsored search auctions | Double Estimator · Sponsored search auctions | incentive compatibility | Mixed |
| add weighted policy (Thompson-like sampling) to Weighted Q-Learning (ablation) | Weighted Q-Learning · Grid world (Gaussian sigma=1) | average reward per step | Improved |
| use Weighted Q-Learning vs ME, DE, MME on noisy grid world | Weighted Q-Learning · Grid world (noisy rewards) | learning speed of best policy | Hurt |
| use WE with infinite actions (GP) instead of binned ME/DE/MME | WE infinity · Pricing problem | bias | Improved |
| use Weighted FQI instead of other FQI variants with discrete actions | Weighted FQI · Swing-Up Pendulum (11 discrete actions) | average reward: less or equal to 50 | Improved |
| use continuous-action WE (infinity) in FQI | Weighted FQI · Swing-Up Pendulum (continuous actions) | average reward | No clear effect |
| use WDQN instead of DDQN and AvgDQN | WDQN · Lunar Lander (stochastic) | sample efficiency and prediction accuracy | Improved |
Closest papers by meaning
- Estimating the Maximum Expected Value in Continuous Reinforcement Learning Problems (AAAI 2017 · similarity 0.91)
- Maxmin Q-learning: Controlling the Estimation Bias of Q-learning (ICLR 2020 · similarity 0.91)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.88)
- Exclusively Penalized Q-learning for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Stochastic Q-learning for Large Discrete Action Spaces (ICML 2024 · similarity 0.88)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
