Research map / Time series / Causal and generative time series
Oscillatory State-Space Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LinOSS, a state-space model built on discretized forced harmonic oscillators with implicit and implicit-explicit parallel scans, proven stable with nonnegative diagonal state matrix and universal for causal continuous operators.
Problem
Efficient and stable learning of long-range interactions in sequence models; state-space models require restrictive structured state matrices to remain stable and expressive.
Approach
Base a state-space model on a system of forced linear second-order ODEs (harmonic oscillators), discretize stably (implicit or implicit-explicit symplectic), and run it with associative parallel scans; stability only requires a nonnegative diagonal state matrix.
Key results
- LinOSS-IM averages 67.8% accuracy over six UEA datasets, versus 64.3% for the next best (Log-NCDE).
- On PPG-DaLiA, LinOSS-IM reaches MSE 6.4e-2 versus 12.17e-2 for LRU and 10.65e-2 for Mamba, nearly a factor of 2 improvement.
- On weather 720-step forecasting, LinOSS-IMEX achieves MAE 0.508 versus 0.578 for S4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LinOSS-IM instead of Log-NCDE | UEA-MTSCA · UEA-MTSCA Avg | accuracy: 64.3 -> 67.8 | Helped |
| Use LinOSS-IM instead of Log-NCDE | PPG-DaLiA · PPG-DaLiA | MSE × 10⁻²: 9.56 ± 0.59 -> 6.4 ± 0.23 | Helped |
| Use LinOSS-IMEX instead of S4 | Weather forecasting · Weather (720-step forecast) | mean absolute error: 0.578 -> 0.508 | Helped |
| Use squared instead of ReLU state-matrix parameterization (ablation) | LinOSS-IM · UEA-MTSCA Avg | accuracy: 67.8 -> 67.1 | Hurt |
| Use squared instead of ReLU state-matrix parameterization (ablation) | LinOSS-IM · UEA-MTSCA datasets | accuracy | Mixed |
| Use standard normal initialization with squared state-matrix parameterization (ablation) | LinOSS-IM · UEA-MTSCA Avg | accuracy: 67.1 -> 64.0 | Hurt |
| Vary the discretization timestep from 10⁻³ to 1 (ablation) | LinOSS-IM · SCP1, Heartbeat, and MotorImagery | accuracy | No clear effect |
| Use conservative LinOSS-IMEX instead of dissipative LinOSS-IM (ablation) | Simple harmonic motion prediction · Energy-conserving simple harmonic motion | test error: more than 8 | Helped |
Surprising findings
- Standard normal initialization of the state matrix fails on EigenWorms with high variance, attributed to small eigenvalues and vanishing gradients.
- LinOSS-IMEX preserves error constant over time on harmonic motion while LinOSS-IM error grows, outperforming it by more than 8x at the end of the interval.
Limitations
- LinOSS-IMEX and LinOSS-IM have different advantages depending on data, so the choice of discretization is task-dependent.
- Higher-order integrators (velocity Verlet) are only sketched and not evaluated.
- Parameter counts and memory of LinOSS-IM and IMEX are sometimes much larger than competitors on some datasets.
Experimental setup
Datasets: UEA time-series classification archive (six long datasets) PPG-DaLiA Weather forecasting dataset Simple harmonic motion (synthetic)
Benchmarks: UEA time-series classification (EigenWorms, SCP1, SCP2, Ethanol, Heartbeat, Motor) PPG-DaLiA Weather 720-step forecasting
Metrics: test accuracy mean squared error mean absolute error GPU memory run time
Baselines: NRDE NCDE Log-NCDE LRU S5 S6 Mamba S4 Informer LogTrans Reformer LSTMa
Builds on: S5 LRU HiPPO Coupled Oscillatory RNNs Log-NCDE
Closest papers by meaning
- BioOSS: A Bio-Inspired Oscillatory State System with Spatio-Temporal Dynamics (NeurIPS 2025 · similarity 0.87)
- Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems (NeurIPS 2024 · similarity 0.86)
- Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction (NeurIPS 2026 · similarity 0.86)
- MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models (NeurIPS 2026 · similarity 0.86)
- Inference of Neural Dynamics Using Switching Recurrent Neural Networks (NeurIPS 2024 · similarity 0.86)
- Mechanistic Interpretability of RNNs emulating Hidden Markov Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.