Research map / Reinforcement learning / Policy gradient and actor-critic
Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods
Seohong Park, Jaekyeom Kim, Gunhee Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Repeat actions until the state leaves a learned safe region instead of using a preset action duration | Policy-gradient methods on MuJoCo environments · Deterministic MuJoCo environments across varying δ | total reward | Improved |
| Use state-based safe regions rather than continuous action durations | SAR-PPO versus FiGAR-C-PPO · Stochastic MuJoCo environments | total reward | Improved |
| Use safe-region action repetition rather than per-step decisions | SAR-PPO versus DAU · Deterministic MuJoCo environments across varying δ | total reward | Improved |
| Use safe-region action repetition | SAR-PPO · Lowest-δ deterministic MuJoCo settings | decision steps: 17.6× fewer decision steps on average | Improved |
| Use safe-region action repetition rather than standard PPO | PPO · Deterministic MuJoCo environments at low δ | total reward | Improved |
| Scale PPO's learning rate by δ / δ₀ | PPO · Deterministic MuJoCo environments at low δ | total reward | No clear effect |
| Use SAR rather than autoregressive action noise | PPO · Deterministic MuJoCo environments at low δ | total reward | Improved |
| Use safe-region action repetition rather than FiGAR-C | A2C · Deterministic MuJoCo environments | total reward | Improved |
| Use safe-region action repetition rather than FiGAR-C | TRPO · Stochastic MuJoCo environments | total reward | Improved |
| Add temporal thresholds to safe-region criteria (ablation) | SAR-PPO · Four stochastic POMDP MuJoCo environments | total reward | Improved |
| Remove the upper limit on action duration (ablation) | SAR · MuJoCo environments | total reward | Mixed |
| Decrease δ while using SAR | SAR · Swimmer-v2 | total reward | Hurt |
Closest papers by meaning
- Reusing Trajectories in Policy Gradients Enables Fast Convergence (ICML 2026 · similarity 0.88)
- Convergence Analysis of Policy Gradient Methods with Dynamic Stochasticity (ICML 2025 · similarity 0.88)
- Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.88)
- Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms (JMLR 2022 · similarity 0.87)
- Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Select before Act: Spatially Decoupled Action Repetition for Continuous Control (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
