Research map / Reinforcement learning / Offline RL
DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty
Mingxuan Cui, Duo Zhou, Yuxuan Han, Grani A. Hanasusanto, Qiong Wang, Huan Zhang, Zhengyuan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add distributionally robust Bellman update | SAC · Pendulum (length +-20%) | average reward: 35% | Improved |
| add distributionally robust Bellman update | SAC · LunarLander (20% perturbation) | average reward: 240 vs under 180 | Improved |
| add distributionally robust Bellman update | SAC · LunarLander (parameters -30%) | reward: 9.8x | Improved |
| add distributionally robust Bellman update | SAC · HalfCheetah (back_damping within 50%) | average reward: 5950 -> 6300 | Improved |
| use functional approximation instead of accurate Bellman operator (ablation) | DR-SAC · robustness environments | robustness and training time: less than 2% training time | Improved |
| use DR-SAC instead of RFQI | offline DR-RL · training time | training time: 93 -> 4 min (Cartpole) | Improved |
| use 5 GD steps instead of RFQI's 1000 (ablation) | RFQI-style update · training efficiency | performance: 1000 -> 5 GD steps | Improved |
| vary VAE latent dimension between 5 and 20 (ablation) | DR-SAC · Pendulum | robustness | No clear effect |
| replace VAE with normalizing flow (ablation) | DR-SAC · Pendulum | robustness | Hurt |
| replace VAE with diffusion model (ablation) | DR-SAC · Pendulum | robustness / training time: at least 4.5x training time | No clear effect |
| reduce RFQI gradient descent steps (ablation) | RFQI · unperturbed environments | performance | Hurt |
Closest papers by meaning
- Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages (ICML 2024 · similarity 0.88)
- Single-Trajectory Distributionally Robust Reinforcement Learning (ICML 2024 · similarity 0.88)
- Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithms (NeurIPS 2024 · similarity 0.88)
- Robust Offline Reinforcement Learning with Linearly Structured $f$-Divergence Regularization (ICML 2025 · similarity 0.88)
- Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
