Research map / Reinforcement learning / Policy gradient and actor-critic
Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove clipped double Q-learning (ablation) | SAC · MuJoCo Walker2d-v2 | return: 5820 -> 1888 | Helped |
| remove clipped double Q-learning (ablation) | SAC · MuJoCo Humanoid-v2 | return: 8081 -> 768 | Helped |
| remove clipped double Q-learning (ablation) | SAC · MuJoCo HalfCheetah-v2 and Swimmer-v2 | return: 15254 -> 15701 | Hurt |
| add clipped double Q-learning (ablation) | MPO and AWAC · MuJoCo | return | No clear effect |
| remove tanh squashing (use MPO action penalty) (ablation) | SAC · MuJoCo Hopper-v2 | return: 3013 -> 6 | Helped |
| add tanh squashing (ablation) | MPO and AWR · MuJoCo | return: 3972 -> 3167 | Mixed |
| add layer normalization (ablation) | SAC · MuJoCo Ant-v2 | return: 5532 -> 7349 | Helped |
| add ELU activation (ablation) | SAC · MuJoCo Swimmer-v2 | return: 114 -> 146 | Helped |
| remove ELU and layer normalization (ablation) | MPO · MuJoCo Ant-v2 | return: 6584 -> 807 | Hurt |
| add ELU and layer normalization (ablation) | AWR · MuJoCo Ant-v2 | return: 1127 -> 2283 | Helped |
| large network (L) instead of middle (M) (ablation) | SAC · MuJoCo Ant-v2 | return: 5532 -> 7495 | Helped |
| middle network (M) instead of large (L) (ablation) | MPO · MuJoCo Hopper-v2 | return: 2136 -> 661 | Hurt |
Closest papers by meaning
- ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages (ICML 2024 · similarity 0.89)
- Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL (ICML 2024 · similarity 0.88)
- Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategies (NeurIPS 2025 · similarity 0.88)
- Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning (ICML 2024 · similarity 0.88)
- TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing (NeurIPS 2026 · similarity 0.88)
- Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.