Research map / Architectures and efficiency / RNNs and normalization

How deep is your network? Deep vs. shallow learning of transfer operators

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Theory of depth and width

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RaNNDy, a randomized neural network with fixed random hidden layers where only the output layer is trained via variational (Rayleigh/VAMP-style) objectives, yielding closed-form eigenfunctions and ensemble-based uncertainty for transfer and Schrödinger operators.

Problem

Learning transfer operators (Koopman, Perron–Frobenius, forward-backward) and their eigen/singular functions from data is expensive and unstable when the dictionary is learned by backpropagation (e.g., VAMPnets), with slow convergence and hyperparameter sensitivity.

Approach

Fix randomly initialized hidden layers as a random feature map and optimize only the output layer; the variational quotient then reduces to a generalized eigenvalue problem, giving a closed-form solution for eigenfunctions and enabling ensemble-based uncertainty estimates.

Key results

• RaNNDy runs in 0.32 s vs 121.35 s for VAMPnets (100 epochs) on the OU process, about two to three orders of magnitude faster.
• On protein folding, RaNNDy takes 0.52 s vs 60.44 s on Chignolin and 1.34 s vs 139.98 s on NuG2, with frame counts closely matching VAMPnets (e.g., 8282 vs 8296 folded frames).
• RaNNDy eigenvalues and eigenfunctions match analytic OU, Hermite-based quantum oscillator solutions, with ensemble variance (100 models) highlighting uncertainty in sparsely sampled regions.

Surprising findings

• The paper notes that VAMPnets may be better when a randomized basis is not optimal, indicating RaNNDy is not uniformly superior in approximation quality.

Limitations

• Accuracy depends on the distribution chosen for random weights and biases; ill-conditioned choices degrade approximation.
• Less flexible than VAMPnets when a random basis is not well suited to the operator.
• Approximation is less accurate near domain boundaries with sparse data.

Experimental setup

Datasets: simulated OU process trajectories lemon-slice and triple-well potential trajectories Bickley jet trajectories generated with deeptime D.E. Shaw Research Chignolin and NuG2 MD trajectories

Benchmarks: Ornstein-Uhlenbeck analytic eigenfunctions lemon-slice potential triple-well potential quantum harmonic oscillator Chignolin NuG2 Bickley jet

Metrics: run time eigenvalue agreement eigenfunction agreement with exact solutions number of identified folded/unfolded frames

Baselines: VAMPnets EDMD

Builds on: EDMD VAMPnets random vector functional link extreme learning machine variational approach to conformation dynamics

Closest papers by meaning

• Langevin-Informed Transfer Learning: Replacing the Target Samples by Black-Box Feedback (NeurIPS 2026 · similarity 0.84)
• STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem (NeurIPS 2025 · similarity 0.83)
• Physics-Informed DeepONets for drift-diffusion on metric graphs: simulation and parameter identification (ICML 2025 · similarity 0.83)
• Free Decompression with Algebraic Spectral Curves (NeurIPS 2026 · similarity 0.83)
• Learning stochastic dynamics from snapshots through regularized unbalanced optimal transport (ICLR 2025 · similarity 0.83)
• Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.83)