Research map / Reinforcement learning / Policy gradient and actor-critic
Belief Projection-Based Reinforcement Learning for Environments with Delayed Feedback
Jangwon Kim, Hangyeol Kim, Jiwook Kang, Jongchan Baek, Soohee Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| BPQL (beta Q-values on original state) vs Augmented SAC, d=3 | SAC · HalfCheetah-v3 | average return: 6054.0 -> 8100.1 | Improved |
| BPQL vs Augmented SAC, d=6 | SAC · HalfCheetah-v3 | average return: 2012.6 -> 6334.6 | Improved |
| BPQL vs Augmented SAC, d=9 | SAC · HalfCheetah-v3 | average return: 1297.2 -> 5887.5 | Improved |
| BPQL vs Delayed-SAC (model-based), d=9 | SAC · Walker2d-v3 | average return: 2.6 -> 4104.3 | Improved |
| BPQL vs Augmented SAC, d=9 | SAC · Hopper-v3 | average return: 1497.8 -> 2993.4 | Improved |
| BPQL vs baselines at short delay d=3 | SAC · Hopper-v3 | average return: 2821.7 -> 2922.5 | No clear effect |
| BPQL vs Augmented SAC on InvertedPendulum, d=6 and 9 | SAC · InvertedPendulum-v2 | average return: 1000.0 -> 983.4 | No clear effect |
| Increase delay from 3 to 9 for Augmented SAC | Augmented SAC · HalfCheetah-v3 | average return: 6054.0 -> 1297.2 | Hurt |
| Increase delay from 3 to 9 for BPQL | BPQL · HalfCheetah-v3 | average return: 8100.1 -> 5887.5 | Hurt |
| Increase Q-network hidden size (256-256 to 512, 1024, 3 layers) (ablation) | Augmented SAC · delayed MuJoCo | average return | No clear effect |
| Add actuator noise (std 0.1, 0.15) to InvertedPendulum | BPQL · noisy InvertedPendulum-v2 | average return | Improved |
| Extremely long delay (20 steps HalfCheetah, 25 InvertedPendulum) | BPQL · HalfCheetah-v3, InvertedPendulum-v2 | average return | Improved |
Closest papers by meaning
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.86)
- Delayed homomorphic reinforcement learning for environments with delayed feedback (NeurIPS 2026 · similarity 0.86)
- Periodic agent-state based Q-learning for POMDPs (NeurIPS 2024 · similarity 0.86)
- Sample Efficient Reinforcement Learning with Partial Dynamics Knowledge (AAAI 2024 · similarity 0.85)
- Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control (ICLR 2025 · similarity 0.85)
- Variational Delayed Policy Optimization (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
