Research map / Architectures and efficiency / Attention and transformers
The Expressive Limits of Diagonal SSMs for State-Tracking
Mehran Shakerinava, Behnoush Khavari, Siamak Ravanbakhsh, Sarath Chandar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| allow negative eigenvalues (Negative Mamba instead of Mamba) | Mamba · Group state-tracking C2 (1-layer) | longest length >=100 with accuracy >90%: fail -> 1000 | Improved |
| stack a second layer on Negative Mamba | Negative Mamba · Group state-tracking C2xC4 | longest length >=100 with accuracy >90%: fail -> 360 | Improved |
| use complex-valued rotations (AUSSM instead of Negative Mamba) | Negative Mamba · Group state-tracking C6 (1-layer) | longest length >=100 with accuracy >90%: fail -> 940 | Improved |
| add a second layer to AUSSM | AUSSM · Group state-tracking S3 | longest length >=100 with accuracy >90%: fail -> fail | No clear effect |
| add a second layer to AUSSM | AUSSM · Group state-tracking C6 | longest length >=100 with accuracy >90%: 940 -> 100 | Hurt |
| add a second layer to Simple AUSSM | Simple AUSSM · Group state-tracking C2 | longest length >=100 with accuracy >90%: 160 -> 1000 | Improved |
| increase group size from C2 to C60 | RNN · Group state-tracking (1-layer) | longest length >=100 with accuracy >90%: 1000 -> fail | Hurt |
| initialize AUSSM near analytical S3 solution (ablation) | AUSSM · Group state-tracking S3 | longest length >=100 with accuracy >90%: fail -> extrapolates at least four times training length | Improved |
Closest papers by meaning
- The Illusion of State in State-Space Models (ICML 2024 · similarity 0.86)
- Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models (NeurIPS 2025 · similarity 0.86)
- Theoretical Foundations of Deep Selective State-Space Models (NeurIPS 2024 · similarity 0.85)
- On the Expressive Power and Limitations of Multi-Layer SSMs (NeurIPS 2026 · similarity 0.85)
- Provable Benefits of Complex Parameterizations for Structured State Space Models (NeurIPS 2024 · similarity 0.85)
- Bridging Expressivity and Scalability with Adaptive Unitary SSMs (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
