Research map / Probabilistic and scientific modeling / Molecules and proteins
Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HiSS, a two-level hierarchy of stacked deep state space models (e.g., S4/Mamba) for continuous sensor-to-sequence prediction, and releases the CSP-Bench benchmark of six sensor datasets.
Problem
Continuous sequence-to-sequence prediction from long, high-frequency raw sensor streams (tactile, IMU) with small labeled datasets, where flat Transformers, LSTMs and SSMs struggle with efficiency and temporal redundancy.
Approach
Stack two SSMs at different temporal resolutions: a shared low-level SSM processes non-overlapping chunks of the sensor sequence into chunk features, and a high-level sequence model maps the downsampled chunk-feature sequence to the output sequence, trained end-to-end with MSE.
Key results
- HiSS with S4 low-level and Mamba high-level reaches 0.5663 MSE on Marker Writing and 0.1316 on Intrinsic Slip, about 35.87% and 25.10% better than the best flat model on those tasks.
- HiSS models improve median MSE by about 23% over the best flat SSM across tasks, while flat SSMs beat LSTMs and Transformers by about 10-14% median MSE.
- On 30% of training data, HiSS outperforms all flat baselines on every task reported in the fraction experiment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace a flat sequence model with hierarchical HiSS | continuous sequence prediction on CSP-Bench · CSP-Bench tasks overall | median MSE: ~23% | Helped |
| use HiSS instead of the best flat model | CSP-Bench models · Marker Writing | MSE: +35.87% | Helped |
| use HiSS instead of the best flat model | CSP-Bench models · Intrinsic Slip | MSE: +25.10% | Helped |
| use HiSS instead of the best flat model | CSP-Bench models · Joystick Control | MSE: +8.10% | Helped |
| use HiSS instead of the best flat model | CSP-Bench models · RoNIN | MSE: +30.74% | Helped |
| use HiSS instead of the best flat model | CSP-Bench models · VECtor | MSE: +21.30% | Helped |
| use HiSS instead of the best flat model | CSP-Bench models · TotalCapture | MSE: -37.36% | Hurt |
| increase chunk size from 5 to 10 (ablation) | HiSS · Marker Writing | MSE: 1.18 -> 0.57 | Helped |
| increase chunk size from 10 to 15 (ablation) | HiSS · Marker Writing | MSE: 0.57 -> 0.54 | Helped |
| increase chunk size from 5 to 10 (ablation) | HiSS · Intrinsic Slip | MSE: 0.20 -> 0.13 | Helped |
| increase chunk size from 10 to 15 (ablation) | HiSS · Intrinsic Slip | MSE: 0.13 -> 0.12 | Helped |
| train on a smaller fraction of the training set | HiSS versus flat models · CSP-Bench tasks | MSE | Helped |
Surprising findings
- On TotalCapture, the flat LSTM outperforms all SSM and HiSS models, and HiSS is 37.36% worse than the best flat model there.
- Simply downsampling sensor inputs to the output frequency does not close the gap with HiSS, suggesting HiSS extracts more information than naive downsampling.
Limitations
- Requires an extra chunk-size hyperparameter; chunk size is only briefly analyzed.
- Underperforms flat LSTM on TotalCapture with high-dimensional inputs and outputs.
- Benchmark is limited to six datasets and a limited set of sensor types.
Experimental setup
Datasets: ReSkin Marker Writing ReSkin Intrinsic Slip XELA Joystick Control RoNIN VECtor TotalCapture
Benchmarks: CSP-Bench
Metrics: MSE median relative MSE improvement
Baselines: Transformer LSTM S4 Mamba MEGA MEGA-chunk
Builds on: S4 Mamba MEGA
Closest papers by meaning
- Accurate and Steady Inertial Pose Estimation through Sequence Structure Learning and Modulation (NeurIPS 2024 · similarity 0.84)
- Trajectory World Models for Heterogeneous Environments (ICML 2025 · similarity 0.83)
- UnHiPPO: Uncertainty-aware Initialization for State Space Models (ICML 2025 · similarity 0.83)
- Gated Inference Network: Inference and Learning State-Space Models (NeurIPS 2024 · similarity 0.83)
- Markovian Dynamics Enforcer: Feasibility Preserving Correction on Learned Dyanmics Manifolds (NeurIPS 2026 · similarity 0.83)
- Identifying latent state transitions in non-linear dynamical systems (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.