Research map / Reinforcement learning / Policy gradient and actor-critic
Parameter-Based Value Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PBVFs (PSSVF, PSVF, or PAVF) instead of a conventional single-policy value function | Off-policy actor-critic on continuous-control tasks · Inverted Pendulum, Swimmer, MountainCarContinuous, Reacher | return | Mixed |
| Use deep-policy PSVF or PAVF | PBVF actor-critic · Swimmer-v3 and MountainCarContinuous | return | Helped |
| Optimize newly initialized policies offline using a trained, frozen PSSVF | PSSVF · Swimmer-v3, zero-shot policies | average return: 225 -> 297 | Helped |
| Train new policies from a PSVF learned offline on fixed fragmented-behavior data | PSVF · Swimmer-v3, offline fragmented-behavior dataset | return: 58 -> 237 | Helped |
| Continue offline policy optimization after the reported peak | PSVF · Swimmer-v3, offline fragmented-behavior dataset | return: 237 -> 167 | Hurt |
| Remove the action-value gradient with respect to policy parameters, ∇θQ(s,a,θ) (ablation) | Deterministic PAVF · Swimmer | return | Hurt |
| Remove the action-value gradient with respect to policy parameters, ∇θQ(s,a,θ) (ablation) | Deterministic PAVF · Hopper | return | Hurt |
| Use standard discounting rather than a larger discount factor (ablation) | TD-based DDPG, PSVF, and PAVF · Swimmer-v3 | return | Hurt |
| Increase the discount factor to make learning more far-sighted (ablation) | TD-based DDPG, PSVF, and PAVF · Swimmer-v3 | return | Helped |
| Remove observation normalization (ablation) | Deterministic PSVF and PAVF · Continuous-control tasks | return | Hurt |
Closest papers by meaning
- Learning Value Functions in Deep Policy Gradients using Residual Variance (ICLR 2021 · similarity 0.91)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.90)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- Efficient Off-Policy Learning for High-Dimensional Action Spaces (ICLR 2025 · similarity 0.90)
- Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic (ICML 2024 · similarity 0.89)
- Value Improved Actor Critic Algorithms (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.