Research map / Reinforcement learning / Policy gradient and actor-critic
Decision S4: Efficient Sequence-Based RL via State Spaces Layers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the transformer sequence model with S4 | Decision Transformer-style offline RL · Mujoco Hopper, HalfCheetah, and Walker2D datasets | normalized reward: DS4 exceeds DT on all nine reported dataset settings | Helped |
| fine-tune the offline DS4 policy on-policy | DS4 · Hopper-medium | normalized reward: 89.47 -> 111.3 | Helped |
| fine-tune the offline DS4 policy on-policy | DS4 · Hopper-replay | normalized reward: 87.67 -> 88.42 | Helped |
| fine-tune the offline DS4 policy on-policy | DS4 · HalfCheetah-medium | normalized reward: 47.31 -> 47.32 | No clear effect |
| fine-tune the offline DS4 policy on-policy | DS4 · HalfCheetah-replay | normalized reward: 43.81 -> 45.17 | Helped |
| fine-tune the offline DS4 policy on-policy | DS4 · Walker2D-medium | normalized reward: 81.36 -> 81.71 | Helped |
| fine-tune the offline DS4 policy on-policy | DS4 · Walker2D-replay | normalized reward: 80.26 -> 76.97 | Hurt |
| reduce the maximum number of recurrent training steps (ablation) | DS4 · Mujoco tasks | normalized reward | Hurt |
| replace the S4 core with an RNN (ablation) | DS4 · HalfCheetah-medium, replay, and expert | normalized reward | Hurt |
| reduce state size N from 64 to 32 or 16 while keeping H=64 (ablation) | DS4 · HalfCheetah-medium, replay, and expert | normalized reward: 47.31 -> 47.01; 47.31 -> 47.42; 43.80 -> 43.74; 43.80 -> 43.61; 94.75 -> 94.03; 94.75 -> 94.44 | No clear effect |
| reduce channel count H from 64 to 32 (ablation) | DS4 · HalfCheetah-replay | normalized reward: 43.80 -> 40.71 | Hurt |
| reduce channel count H from 64 to 16 (ablation) | DS4 · HalfCheetah-replay | normalized reward: 43.80 -> 25.97 | Hurt |
Closest papers by meaning
- Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control (ICLR 2025 · similarity 0.89)
- Rethinking Decision Transformer via Hierarchical Reinforcement Learning (ICML 2024 · similarity 0.89)
- Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Rate (NeurIPS 2024 · similarity 0.89)
- Critic-Guided Decision Transformer for Offline Reinforcement Learning (AAAI 2024 · similarity 0.88)
- Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL (NeurIPS 2024 · similarity 0.88)
- In-Context Reinforcement Learning From Suboptimal Historical Data (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.