Research map / Architectures and efficiency / Attention and transformers
Diagonal State Spaces are as Effective as Structured State Spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a diagonal state-space parameterization instead of S4's diagonal-plus-low-rank parameterization | State-space sequence models · LRA average | accuracy: 80.21 vs 81.88 | Helped |
| Use a diagonal state-space parameterization instead of S4 | Raw Speech Commands classification · Speech Commands Raw | accuracy: 98.1 vs 98.2 | No clear effect |
| Replace Skew-Hippo initialization with random initialization of Lambda (ablation) | DSS-softmax · LRA and Speech Commands | accuracy: 60.6 -> 57.6; 84.8 -> 80.54; 85.7 -> 72.13; 84.6 -> 77.5; 97.7 -> 96.8 | Hurt |
| Limit the kernel length to 128 (ablation) | DSS-softmax · LRA and Speech Commands | accuracy: 60.6 -> 51.6; 84.8 -> 75.41; 85.7 -> 83.8; 84.6 -> 65.1; 97.7 -> 96.7 | Mixed |
| Omit the scale term Lambda^-1 (e^(Lambda Delta) - I) from the DSS-exp kernel (ablation) | DSS-exp · LRA | accuracy: 81.18 -> 65.03 average | Hurt |
Closest papers by meaning
- Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing (ICLR 2025 · similarity 0.86)
- Mamba Modulation: On the Length Generalization of Mamba Models (NeurIPS 2025 · similarity 0.86)
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (ICML 2024 · similarity 0.86)
- Block-Biased Mamba for Long-Range Sequence Processing (NeurIPS 2025 · similarity 0.86)
- Generalization Error Analysis for Selective State-Space Models Through the Lens of Attention (NeurIPS 2025 · similarity 0.85)
- Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.