Research map / Reinforcement learning / Policy gradient and actor-critic
Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic Methods
Jeong Woon Lee, Kyoleen Kwak, Daeho Kim, Hyoseok Hwang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PAVE critic regularization (no actor change) | TD3 · Hopper | smoothness score (lower better): 2.715 -> 0.950 | Improved |
| add PAVE critic regularization | TD3 · Walker2d | cumulative return: 4834 -> 5563 | Improved |
| add PAVE critic regularization | SAC · Reacher | smoothness score (lower better): 0.053 -> 0.052 | No clear effect |
| add PAVE critic regularization | SAC · Walker2d | smoothness score (lower better): 0.748 -> 0.584 | Improved |
| add Curvature Preservation loss to MPR + VFC (ablation) | PAVE (TD3) · Walker2d | cumulative return: 4648.2 -> 5562.9 | Improved |
| add VFC to MPR (ablation) | PAVE (TD3) · LunarLander | smoothness score (lower better): 1.748 -> 1.177 | Improved |
| add MPR alone to base TD3 (ablation) | TD3 · LunarLander | smoothness score (lower better): 1.809 -> 1.748 | No clear effect |
| add Curv to MPR + VFC (ablation) | PAVE (TD3) · Walker2d (factorial) | smoothness score (lower better): 2.095 -> 1.483 | Improved |
| add critic spectral normalization (ablation) | TD3 critic · LunarLander | cumulative return: 228 -> −970 | Hurt |
| add critic gradient penalty (ablation) | TD3 critic · LunarLander | smoothness score (lower better): 1.81 -> 2.18 | Hurt |
| add actor-side CAPS on top of PAVE (ablation) | PAVE (TD3) · LunarLander | smoothness score (lower better): 0.541 -> 0.543 | No clear effect |
| add PAVE to enable learning | TD3 · Humanoid-v5 | cumulative return: 309 -> 2894 | Improved |
Closest papers by meaning
- Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (ICLR 2025 · similarity 0.89)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.88)
- Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation (ICML 2024 · similarity 0.88)
- Mean Field Langevin Actor-Critic: Faster Convergence and Global Optimality beyond Lazy Learning (ICML 2024 · similarity 0.88)
- Actor-Critics Can Achieve Optimal Sample Efficiency (ICML 2025 · similarity 0.88)
- Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
