Research map / Probabilistic and scientific modeling / Dynamical systems
Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AL-RNNs, a PLRNN variant with only P of M units using ReLUs, trained with identity teacher forcing, to automatically yield minimal piecewise-linear symbolic codings of chaotic attractors (Lorenz-63, Rössler) and empirical ECG/fMRI data.
Problem
Piecewise-linear RNN reconstructions of dynamical systems typically need very many linear subregions (2^M), making mathematical analysis of learned dynamics intractable; switching linear models are hard to train for DSR.
Approach
Use a PLRNN in which only a subset of P units has ReLU nonlinearities while the rest are linear, so the number of linear subregions is 2^P rather than 2^M; the partition into subregions gives a provably topology-preserving symbolic coding, and training via identity teacher forcing finds minimal PWL representations automatically.
Key results
- On Lorenz-63 the AL-RNN with id-TF reaches D_stsp 0.34±0.05 vs 0.26±0.03 for shPLRNN+GTF, and on ECG AL-RNN achieves D_stsp 3.0±0.7 vs 4.3±0.6 for shPLRNN+GTF (best among compared methods).
- Minimal reconstructions use 3 linear subregions for Lorenz-63 and 2 for Rössler, matching known topologically minimal PWL forms; for Rössler 4 of 45 subregions cover 80% of data.
- On fMRI data with P=2, the four linear subregions aligned with task stages with average classification accuracy 0.78 ± 0.05.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AL-RNN + id-TF instead of SINDy | Lorenz-63 reconstruction · Lorenz-63 | D_stsp: 0.24\pm0.00 -> 0.34\pm0.05 | Hurt |
| Use AL-RNN + id-TF instead of shPLRNN + GTF | Lorenz-63 reconstruction · Lorenz-63 | D_H: 0.090\pm0.007 -> 0.081\pm0.012 | Helped |
| Use AL-RNN + id-TF instead of shPLRNN + GTF | ECG reconstruction · ECG | D_stsp: 4.3\pm0.6 -> 3.0\pm0.7 | Helped |
| Use AL-RNN + id-TF instead of shPLRNN + GTF | ECG reconstruction · ECG | D_H: 0.34\pm0.02 -> 0.29\pm0.04 | Helped |
| Use AL-RNN + id-TF instead of SINDy | Lorenz-96 reconstruction · Lorenz-96 | D_stsp: 1.59 \pm 0.00 -> 1.64\pm0.03 | Hurt |
| Use AL-RNN + id-TF instead of SINDy | Lorenz-96 reconstruction · Lorenz-96 | D_H: 0.06\pm0.00 -> 0.089\pm0.001 | Hurt |
| Use AL-RNN + id-TF instead of shPLRNN + GTF | EEG reconstruction · EEG | D_stsp: 2.1\pm0.2 -> 2.6\pm0.3 | Hurt |
| Use AL-RNN + id-TF instead of shPLRNN + GTF | EEG reconstruction · EEG | D_H: 0.11\pm0.01 -> 0.14\pm0.03 | Hurt |
| Increase the number of PWL units from too few to a sufficient number (ablation) | AL-RNN · Lorenz-63 and Rössler | reconstruction performance | Helped |
| Increase PWL units beyond the performance threshold (ablation) | AL-RNN · Lorenz-63 and Rössler | reconstruction performance | No clear effect |
| Add linear units while the number of PWL units is insufficient (ablation) | AL-RNN · Lorenz-63 | reconstruction performance | No clear effect |
| Add linear units when the number of PWL units is sufficient (ablation) | AL-RNN · Lorenz-63 | reconstruction performance | Helped |
Surprising findings
- Trained AL-RNNs occupy only a small fraction of the 2^P possible subregions, with saturation well below the theoretical limit.
- Reconstructions with the minimal number of PWL units were highly consistent across training runs, unlike standard PLRNNs.
Limitations
- Optimal number of PWL units P must be selected by performance curves or regularization rather than inferred automatically.
- Not all empirical dynamical systems may admit topologically minimal low-subregion representations.
- Symbolic coding ignores subregion borders and proximity matching shows only approximate segmentation for observed attractors.
Experimental setup
Datasets: Lorenz-63 Rössler PPG-DaLiA ECG human fMRI cognitive task data Lorenz-96 EEG motor/imagery data
Benchmarks: Lorenz-63 Rössler ECG Lorenz-96 EEG
Metrics: D_stsp (KL divergence of state space) D_H (Hellinger distance of power spectra) maximum Lyapunov exponent classification accuracy for task stages
Baselines: shPLRNN + GTF dendPLRNN + id-TF Reservoir Computer LSTM-TBPTT SINDy Neural-ODE Long Expressive Memory
Builds on: PLRNN shPLRNN sparse teacher forcing generalized teacher forcing RAdam
Closest papers by meaning
- Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction (NeurIPS 2026 · similarity 0.89)
- Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction (ICML 2024 · similarity 0.89)
- Out-of-Domain Generalization in Dynamical Systems Reconstruction (ICML 2024 · similarity 0.89)
- A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems (NeurIPS 2026 · similarity 0.89)
- Identifying latent state transitions in non-linear dynamical systems (ICLR 2025 · similarity 0.88)
- Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.