Research map / Architectures and efficiency / RNNs and normalization
On the Parameterization and Initialization of Diagonal State Space Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the diagonal S4D-Inv initialization instead of S4-LegS | S4D · LRA average | accuracy: 86.09 -> 85.50 | Hurt |
| Use S4D-Inv instead of S4-LegS initialization | S4D · Speech Commands 16000Hz | accuracy: 95.83 (0.14) -> 96.18 (0.27) | Helped |
| Train B rather than freeze it (ablation) | S4D-Inv · sCIFAR | accuracy: 84.78 (0.16) -> 85.37 (0.43) | Helped |
| Replace Vandermonde computation with softmax computation (ablation) | diagonal SSM · sCIFAR | accuracy: 84.78 (0.16) vs 85.04 (0.22) | No clear effect |
| Replace bilinear discretization with ZOH (ablation) | S4D · sCIFAR | accuracy: 85.20 (0.18) vs 85.02 (0.24) | No clear effect |
| Remove the negative-real-part constraint on A (ablation) | S4D · sCIFAR | accuracy: 85.20 (0.18) -> 85.35 (0.27) | No clear effect |
| Scale imaginary eigenvalue components by 0.01 or 100.0 (ablation) | S4D-Inv and S4D-Lin · sCIFAR, Speech Commands, and BIDMC | performance | Hurt |
| Randomize the imaginary eigenvalue components instead of using uniform spacing (ablation) | S4D-Inv and S4D-Lin · ablation datasets | performance | Hurt |
| Randomize the real eigenvalue components instead of fixing them to -1/2 (ablation) | S4D · ablation datasets | performance | Hurt |
| Randomize both real and imaginary eigenvalue components (ablation) | S4D · ablation datasets | performance | Hurt |
| Use the S4D-Inv2 eigenvalue scaling instead of S4D-Inv (ablation) | S4D · ablation datasets | performance | Hurt |
| Use quadratic imaginary-frequency scaling instead of S4D-Inv or S4D-Lin (ablation) | S4D · ablation datasets | performance | Hurt |
Closest papers by meaning
- Uncovering the Spectral Bias in Diagonal State Space Models (NeurIPS 2025 · similarity 0.90)
- Autocorrelation Matters: Understanding the Role of Initialization Schemes for State Space Models (ICLR 2025 · similarity 0.88)
- Provable Benefits of Complex Parameterizations for Structured State Space Models (NeurIPS 2024 · similarity 0.88)
- HOPE for a Robust Parameterization of Long-memory State Space Models (ICLR 2025 · similarity 0.87)
- From Layers to States: A State Space Model Perspective to Deep Neural Network Layer Dynamics (ICLR 2025 · similarity 0.86)
- From Generalization Analysis to Optimization Designs for State Space Models (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.