Research map / Reinforcement learning / Policy gradient and actor-critic
Clipped Q-Learning: Your Value Clipping Is Secretly A Robust Operator
Zhishuai Liu, Pan Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add moderate value clipping (lambda 0.4 or 0.6) to UCB-Hoeffding Q-learning (ablation) | Algorithm 1 (clipped Q-learning) · five-state simulated MDP, perturbed targets | average cumulative return | Improved |
| add value clipping to Double DQN | Double DQN · Inverted Pendulum with changed pole length | average cumulative reward | Improved |
| add value clipping to Double DQN, tested on nominal pole length 0.6 | Double DQN · Inverted Pendulum nominal environment | average cumulative reward | No clear effect |
| add value clipping to DQN instead of Double DQN (ablation) | DQN · Inverted Pendulum | robustness | No clear effect |
| add value clipping to Double DQN (ablation) | Double DQN · Inverted Pendulum | convergence speed | Improved |
Closest papers by meaning
- Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error (ICML 2024 · similarity 0.88)
- Action Candidate Based Clipped Double Q-learning for Discrete and Continuous Action Tasks (AAAI 2021 · similarity 0.87)
- Epistemic Bellman Operators (AAAI 2025 · similarity 0.85)
- To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions (ICLR 2025 · similarity 0.84)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.84)
- Inverting the Bellman Equation: From $Q$-Values to World Models (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
