Research map / Architectures and efficiency / RNNs and normalization

Scalable Mechanistic Neural Networks

ICLR 2025 · Architectures and efficiency › RNNs and normalization · approach: RNN training dynamics and gradients

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Scalable Mechanistic Neural Networks (S-MNN), a reformulation of MNN that reduces time and space complexity from cubic and quadratic to linear in sequence length for long scientific time-series.

Problem

Original Mechanistic Neural Networks (MNN) have cubic time and quadratic space complexity in sequence length, limiting their use on long temporal sequences in scientific machine learning.

Approach

Reformulates the MNN so that computational time and space scale linearly with sequence length instead of cubically and quadratically, keeping it usable for long-sequence scientific ML tasks.

Key results

• Reduces MNN time complexity from cubic to linear in sequence length and space complexity from quadratic to linear.

Experimental setup

Baselines: Mechanistic Neural Network

Builds on: Mechanistic Neural Networks

Closest papers by meaning

• Mechanistic Neural Networks for Scientific Machine Learning (ICML 2024 · similarity 0.88)
• Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction (NeurIPS 2026 · similarity 0.84)
• Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.84)
• Scalable Neural Network Kernels (ICLR 2024 · similarity 0.84)
• Models of Heavy-Tailed Mechanistic Universality (ICML 2025 · similarity 0.83)
• Oscillatory State-Space Models (ICLR 2025 · similarity 0.83)