Research map / Reinforcement learning / Offline RL
Distributional Soft Actor-Critic With Three Refinements
Jingliang Duan, Wenxuan Wang, Liming Xiao, Jiaxin Gao, Shengbo Eben Li, Chang Liu, Ya-Qin Zhang, Bo Cheng, Keqiang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply the three refinements (DSAC-T) to DSACv1 | DSAC · Humanoid-v3 | average final return: 9074 -> 10829 | Improved |
| apply the three refinements (DSAC-T) to DSACv1 | DSAC · BipedalWalker-hardcore-v3 | average final return: -31 -> 77 | Improved |
| use DSAC-T instead of SAC | SAC · Humanoid-v3 | average final return: 9335 -> 10829 | Improved |
| use DSAC-T instead of SAC | SAC · Swimmer-v3 | average final return: 140 -> 138 | No clear effect |
| add twin value distribution learning (lower-Q distribution) to DSAC | DSAC · Humanoid-v3 | Q-value estimation bias: 207.25 -> -42.29 | Improved |
| add twin value distribution learning (lower-Q distribution) to DSAC | DSAC · Hopper-v3 | Q-value estimation bias: 2171.96 -> -7.00 | Improved |
| use DSAC-T instead of RAC-SAC | RAC-SAC · Humanoid-v3 | Q-value estimation bias: 97.78 -> -42.29 | Improved |
| use DSAC-T instead of RAC-SAC | RAC-SAC · Ant-v3 | Q-value estimation bias: -18.82 -> -10.55 | Improved |
| add expected value substituting to DSAC-T (ablation) | DSAC-T · Humanoid-v3 | learning stability and performance | Improved |
| add twin value distribution learning to DSAC-T (ablation) | DSAC-T · Humanoid-v3 | learning stability and performance | Improved |
| add variance-based critic gradient adjustment to DSAC-T (ablation) | DSAC-T · HalfCheetah-v3 (reward scales 0.01 and 0.1) | reward-scale robustness | Improved |
| use DSAC-T instead of DSACv1 in compute cost | DSAC · Humanoid-v3 | seconds per 1000 iterations: 29.00 -> 35.51 | Hurt |
Closest papers by meaning
- Conservative Offline Distributional Reinforcement Learning (NeurIPS 2021 · similarity 0.90)
- DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty (ICLR 2026 · similarity 0.90)
- A Self-Tuning Actor-Critic Algorithm (NeurIPS 2020 · similarity 0.88)
- Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning (ICLR 2025 · similarity 0.88)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.87)
- Finite-Time Convergence and Sample Complexity of Actor-Critic Multi-Objective Reinforcement Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
