Research map / Reinforcement learning / Policy gradient and actor-critic
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use off-policy SAC instead of DDPG | Continuous control · Ant-v1, Humanoid-v1, Humanoid (rllab) | average return | Helped |
| Use off-policy SAC instead of PPO | Continuous control · More high-dimensional and complex tasks | learning speed | Helped |
| Use SAC instead of soft Q-learning | Maximum-entropy continuous control · Continuous control benchmarks | asymptotic performance | Helped |
| Use a stochastic, entropy-maximizing policy instead of a deterministic policy (ablation) | SAC · Humanoid (rllab) | stability across random seeds | Helped |
| Use two Q-functions instead of one (ablation) | SAC · Harder continuous-control tasks | training speed | Helped |
| Remove the separate value network (ablation) | SAC · Continuous control | training stability | Hurt |
| Evaluate using the policy mean rather than stochastic actions (ablation) | SAC · Continuous control | evaluation return | Helped |
| Use very small reward magnitudes (ablation) | SAC · Continuous control | learning performance | Hurt |
| Use very large reward magnitudes (ablation) | SAC · Continuous control | learning performance | Hurt |
| Use a large target value smoothing coefficient (ablation) | SAC · Ant-v1 | training stability | Hurt |
| Use a small target value smoothing coefficient (ablation) | SAC · Ant-v1 | learning speed | Hurt |
| Replace exponentially averaged target updates with periodic hard updates (ablation) | SAC · Continuous-control benchmarks | average return | Mixed |
Closest papers by meaning
- S$2$AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic (ICLR 2024 · similarity 0.90)
- Maximum Entropy Reinforcement Learning with Diffusion Policy (ICML 2025 · similarity 0.90)
- Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL (ICML 2024 · similarity 0.89)
- Generative Actor-Critic with Soft Bridge Policies (NeurIPS 2026 · similarity 0.89)
- RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning (ICML 2024 · similarity 0.89)
- ACE: Off-Policy Actor-Critic with Causality-Aware Entropy Regularization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.