Research map / Reinforcement learning / Policy gradient and actor-critic
Continuous Deep Q-Learning in Optimal Control Problems: Normalized Advantage Functions Analysis
Anton Plaksin, Stepan Martyanov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace NAF's quadratic family (vertex constrained inside action set) with the wider family allowing unconstrained vertex (Bounded NAF) | NAF · Van der Pol, Pendulum, Dubins car, A target problem | learning curve return | Mixed |
| fix P(t,x) to r(t,x)*dt instead of learning it (Reward-based BNAF) (ablation) | BNAF · Van der Pol, Pendulum, Dubins car, A target problem | learning curve return and stability | Improved |
| additionally replace the mu-tilde network by 1/2 r^-1 g^T grad V (Gradient-based BNAF) (ablation) | RB-BNAF · Van der Pol, Pendulum, Dubins car, A target problem | learning curve return | Mixed |
| use plain NAF with tanh policy output | NAF · Pendulum | learning curve return | Mixed |
| use DDPG as baseline versus BNAF variants | DDPG · four control problems | learning curve return | Hurt |
| add a small quadratic control cost term r to problems whose cost does not depend on control | BNAF variants · Pendulum, A target problem | performance | No clear effect |
Closest papers by meaning
- Continuous Deep Q-Learning with Model-based Acceleration (ICML 2016 · similarity 0.92)
- Regularized Q-Learning (NeurIPS 2024 · similarity 0.86)
- A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation (NeurIPS 2024 · similarity 0.86)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.85)
- Universal Approximation Theorem of Deep Q-Networks (ICML 2025 · similarity 0.84)
- Continuous Q-Score Matching: Diffusion Guided Reinforcement Learning for Continuous-Time Control (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
