Research map / Architectures and efficiency / RNNs and normalization
An Expressive and Self-Adaptive Dynamical System for Efficient Function Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EADS, a hierarchical, heterogeneous electronic dynamical system with an on-device training method (EC-Train) that uses intrinsic electrical error signals, evaluated on spatio-temporal graph prediction, PDE solving, and GPT-2 attention kernel replacement.
Problem
Existing physically embodied dynamical systems for learning have low expressivity (flat, homogeneous linear interactions) and high training cost because training runs on digital processors rather than on the device.
Approach
Adds a hierarchical input-hidden-hidden-output structure with nonlinear (ReLU-like) inter-hidden coupling to the real-valued Hamiltonian dynamics, and trains all parameters on-device using the system's own electrical error currents, with the adjoint sensitivity method for hidden-to-hidden weights.
Key results
- EADS reduces test MAE versus the best baseline on all six spatio-temporal datasets, with an average 23.91% reduction and up to 16.29% on Texas COVID.
- EADS reports average training speedup of about 300x over baselines (356x vs NP-GL) and about 1000x inference latency speedup over GPU baselines on graph tasks.
- Replacing GPT-2 attention kernels with EADS raises LAMBADA perplexity only 1.54-1.95 points over the original 35.13, versus NP-GL's 37.79-38.79.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace NP-GL with EADS | Spatial-temporal prediction · PEMS04 | MAE: 17.07 -> 16.92 | Helped |
| Replace NP-GL with EADS | Spatial-temporal prediction · PEMS08 | MAE: 13.51 -> 13.43 | Helped |
| Replace NP-GL with EADS | Spatial-temporal prediction · PM2.5 | MAE: 1.62 -> 1.53 | Helped |
| Replace NP-GL with EADS | Spatial-temporal prediction · PM10 | MAE: 1.73 -> 1.62 | Helped |
| Replace NP-GL with EADS | Spatial-temporal prediction · NYC Taxi | MAE: 3.03 -> 2.46 | Helped |
| Replace NP-GL with EADS | Spatial-temporal prediction · Texas Covid | MAE: 22.04 -> 18.45 | Helped |
| Replace FNO with EADS | PDE solver · Burgers S1=256 | test MAE: 1.41 × 10−5 -> 1.32 × 10−5 | Helped |
| Replace FNO with EADS | PDE solver · Burgers S2=1024 | test MAE: 1.42 × 10−5 -> 1.46 × 10−5 | Hurt |
| Replace FNO with EADS | PDE solver · Burgers S3=4096 | test MAE: 1.53 × 10−5 -> 1.51 × 10−5 | Helped |
| Replace FNO with EADS | PDE solver · Darcy Flow S1=16×16 | test MAE: 4.72 × 10−6 -> 4.31 × 10−6 | Helped |
| Replace FNO with EADS | PDE solver · Darcy Flow S2=32×32 | test MAE: 3.36 × 10−6 -> 3.17 × 10−6 | Helped |
| Replace FNO with EADS | PDE solver · Darcy Flow S3=64×64 | test MAE: 1.03 × 10−5 -> 1.12 × 10−5 | Hurt |
Limitations
- Dynamical-system results are from a simulator (custom CUDA FEA on BRIM embodiment), not fabricated hardware.
- On Darcy Flow at 64x64, EADS (1.12e-5) is worse than FNO (1.03e-5), and on Burgers S3 it is near FNO; accuracy is not uniformly best.
- LLM experiment replaces one attention kernel at a time and degrades perplexity by about 1.5-2 points relative to GPT-2.
Experimental setup
Datasets: PEMS04 PEMS08 PM2.5 PM10 NYC Taxi Texas COVID Burgers equation Darcy Flow LAMBADA
Benchmarks: PEMS04 PEMS08 PM2.5 PM10 NYC Taxi Texas COVID Burgers equation Darcy Flow LAMBADA
Metrics: MAE perplexity training time inference latency
Baselines: Graph WaveNet MTGNN DDGCRN MegaCRN NP-GL NN FCN GNO FNO
Builds on: NP-GL BRIM Adjoint Sensitivity Method Fourier Neural Operator
Closest papers by meaning
- Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians (NeurIPS 2025 · similarity 0.84)
- eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling (NeurIPS 2024 · similarity 0.83)
- Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context (ICML 2024 · similarity 0.83)
- Optimality and NP-Hardness of Transformers in Learning Markovian Dynamical Functions (NeurIPS 2025 · similarity 0.83)
- F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning (NeurIPS 2025 · similarity 0.82)
- OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.