Research map / Probabilistic and scientific modeling / Dynamical systems
Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that geometry-based pruning of RNNs for dynamical systems reconstruction yields sparse, hub-plus-small-world topologies that can be reproduced by a new GeoHub graph-generation algorithm, matching fully connected performance when used as a fixed mask.
Problem
Parameter pruning of RNNs for dynamical systems reconstruction (DSR) is ineffective with magnitude criteria, and the role of network topology in DSR is unexplored. The paper asks how to obtain parsimonious, sparse RNNs for DSR.
Approach
Geometric pruning removes weights whose removal least changes the attractor geometry (KL divergence in state space) rather than weights with small magnitude; the resulting topology (hub-like with small-world features) is extracted and turned into a generator (GeoHub) that produces sparse masks used as fixed topological priors for training PLRNNs from scratch.
Key results
- Geometric pruning reduced parameters by up to 95% for some systems, with sometimes about 5% of weights sufficient, significantly better than random pruning in 10/10 comparisons (p<0.03), while magnitude pruning was not distinguishable from random in 5/10 comparisons.
- Redrawing weights from scratch with a fixed geometric-pruned mask gave similar DSR performance to the original initialization, indicating topology rather than weights drives performance.
- GeoHub topology outperformed Watts-Strogatz, Barabási-Albert and Erdős–Rényi graphs on Lorenz-63, bursting neuron and ECG, and trained faster.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use geometric pruning instead of random pruning | PLRNNs · DS benchmarks | D_stsp and D_H reconstruction quality | Helped |
| Use magnitude-based pruning instead of random pruning (ablation) | PLRNNs · DS benchmarks | D_stsp and D_H reconstruction quality | No clear effect |
| Use GeoHub topology instead of Watts–Strogatz, Barabási–Albert, or Erdős–Rényi topology | RNNs · Lorenz-63, bursting neuron, and ECG | DS reconstruction quality | Helped |
| Use Watts–Strogatz topology instead of Erdős–Rényi topology | RNNs · DS reconstruction benchmarks | DS reconstruction quality | Hurt |
| Resample weights while keeping the geometric-pruning mask fixed (ablation) | Geometry-pruned networks · DS reconstruction benchmarks | DS reconstruction performance | No clear effect |
| Remove connections by weight magnitude rather than geometric contribution (ablation) | RNNs · DS reconstruction benchmarks | Network size at comparable reconstruction quality | Hurt |
| Use geometric pruning on LSTMs instead of magnitude-based pruning | LSTMs · DS reconstruction benchmarks | Reconstruction quality under pruning | Helped |
Surprising findings
- Magnitude-based pruning performed essentially no better than random pruning for DSR, since low-magnitude weights can strongly influence attractor dynamics.
- A pure small-world (Watts-Strogatz) topology diminished DSR performance relative to Erdős–Rényi graphs.
Limitations
- Geometric pruning is computationally costly and mainly serves as an analysis tool rather than a practical method.
- Results are shown for a limited set of autonomous benchmark systems and mainly for PLRNNs, with LSTM and vanilla RNN as secondary checks.
- Ability to generalize beyond natural-science DSR domains (e.g., NLP) is untested.
Experimental setup
Datasets: Lorenz-63 Lorenz-96 bursting neuron model Rössler system PPG-DaLiA ECG
Benchmarks: Lorenz-63 Lorenz-96 bursting neuron Rössler ECG Lorenz-63 with 25% noise
Metrics: D_stsp (KL divergence of state-space occupation) D_H (Hellinger distance of power spectra) 20-step-ahead prediction error average path length clustering coefficient small-world index
Baselines: magnitude-based pruning random pruning Watts-Strogatz graph Barabási-Albert graph Erdős–Rényi graph fully connected RNN
Builds on: PLRNN sparse teacher forcing lottery ticket hypothesis Barabási-Albert model Watts-Strogatz model
Closest papers by meaning
- Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction (NeurIPS 2024 · similarity 0.89)
- Out-of-Domain Generalization in Dynamical Systems Reconstruction (ICML 2024 · similarity 0.89)
- Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data (ICLR 2025 · similarity 0.88)
- A scalable generative model for dynamical system reconstruction from neuroimaging data (NeurIPS 2024 · similarity 0.87)
- Mechanistic Interpretability of RNNs emulating Hidden Markov Models (NeurIPS 2025 · similarity 0.86)
- Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.