Research map / Reinforcement learning / Policy gradient and actor-critic
Correcting discount-factor mismatch in on-policy policy gradient methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add averaging correction to BAC | BAC · Two-state counterexample | probability of choosing optimal action | Helped |
| Leave the state distribution uncorrected | BAC · Two-state counterexample | probability of choosing optimal action | Hurt |
| Add averaging correction to BAC | BAC · Discrete Reacher | squared bias of state emphasis | Helped |
| Add averaging correction to BAC | BAC · Discrete Reacher | variance of state emphasis | Helped |
| Add averaging correction to BAC | BAC · CartPole | undiscounted return | Helped |
| Add averaging correction to BAC | BAC · Acrobot | undiscounted return | No clear effect |
| Add averaging correction to BAC | BAC · MountainCar Continuous | undiscounted return | Helped |
| Add existing γ^t correction | PPO · MuJoCo tasks | undiscounted return | Hurt |
| Add averaging correction | PPO · MuJoCo tasks | undiscounted return | No clear effect |
| Add averaging correction | PPO · Mountain Car Continuous | undiscounted return | Helped |
| Add averaging correction | PPO · Point Mass | undiscounted return | Helped |
| Add averaging correction | PPO · UR-Reacher-2 | return | Helped |
Closest papers by meaning
- A Temporal-Difference Approach to Policy Gradient Estimation (ICML 2022 · similarity 0.91)
- Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning (NeurIPS 2017 · similarity 0.88)
- Relaxed Stationary Distribution Correction Estimation for Improved Offline Policy Optimization (AAAI 2024 · similarity 0.86)
- ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update (ICLR 2024 · similarity 0.85)
- On the Second-Order Convergence of Biased Policy Gradient Algorithms (ICML 2024 · similarity 0.85)
- SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.