Research map / Architectures and efficiency / RNNs and normalization
The Curious Case of In-Training Compression of State Space Models
Makram Chahine, Philipp Nazari, Daniela Rus, T. Konstantin Rusch
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add in-training balanced truncation to state dim 57 | LRU · CIFAR10 | top-3 accuracy (%): 78.2 -> 84.4 | Improved |
| add in-training balanced truncation to state dim 93 | LRU · CIFAR10 | top-3 accuracy (%): 81.8 -> 85.7 | Improved |
| add in-training truncation to state dim 57 | LRU · ListOps | top-3 accuracy (%): 43.4 -> 48.3 | Improved |
| add in-training truncation to state dim 13 | LRU · sMNIST | top-3 accuracy (%): 92.6 -> 95.9 | Improved |
| add in-training truncation to state dim 54 | LRU · AAN | top-3 accuracy (%): 87.5 -> 87.2 | No clear effect |
| add in-training truncation to state dim 95 | LRU · IMDB | top-3 accuracy (%): 82.7 -> 82.2 | No clear effect |
| add in-training truncation to state dim 35 | LRU · Pathfinder | top-3 accuracy (%): 97.3 -> 96.6 | Hurt |
| use CompreSSM instead of HNN regularization at n=13 | LRU · sMNIST | top-3 accuracy (%): 91.7 -> 95.9 | Improved |
| use CompreSSM instead of HNN regularization | LRU · sMNIST | training speedup (x): 0.06 -> 2.8 | Improved |
| use CompreSSM instead of knowledge distillation at n=57 | LRU · CIFAR10 | top-3 accuracy (%): 84.4 -> 79.4 | Hurt |
| use knowledge distillation instead of CompreSSM at n=327 | LRU · CIFAR10 | top-3 accuracy (%): 86.1 -> 87.0 | Improved |
| use CompreSSM instead of KD | LRU · CIFAR10 | training speedup (x): 0.55 -> 1.58 | Improved |
Closest papers by meaning
- HOPE for a Robust Parameterization of Long-memory State Space Models (ICLR 2025 · similarity 0.92)
- Autocorrelation Matters: Understanding the Role of Initialization Schemes for State Space Models (ICLR 2025 · similarity 0.90)
- Elastic Spectral State Space Models for Train-Once Budgeted Inference (NeurIPS 2026 · similarity 0.90)
- Hankel Singular Value Regularization for Highly Compressible State Space Models (NeurIPS 2025 · similarity 0.90)
- MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models (NeurIPS 2026 · similarity 0.89)
- Bridging Expressivity and Scalability with Adaptive Unitary SSMs (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
