Research map / Probabilistic and scientific modeling / Differential equations and PDEs

On the Benefits of Memory for Modeling Time-Dependent PDEs

ICLR 2025 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Neural PDE solvers · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows theoretically and empirically that memory of past states helps neural operators for time-dependent PDEs when observations are low-resolution or noisy, and introduces MemNO (FFNO + S4 memory layer) plus a high-frequency KS benchmark.

Problem

Neural solution operators for time-dependent PDEs are usually Markovian, but it is unclear when conditioning on past states helps; partial observation (low resolution, noise) makes the dynamics non-Markovian per Mori-Zwanzig theory.

Approach

Motivated by Mori-Zwanzig, interleave a causal sequence model (S4) acting on hidden representations of past timesteps into a Markovian neural operator (FFNO), yielding MemNO/S4FFNO; a linear PDE example proves memory can be arbitrarily better than Markovian solutions under Fourier truncation.

Key results

• On KS at resolution 32 with viscosity 0.075, S4FFNO achieves nRMSE 0.139 versus 0.500 for FFNO (about 6x less error on that setting, as reported).
• At resolution 128, FFNO and S4FFNO perform similarly (e.g., 0.006 vs 0.008 for KS nu=0.075), so memory does not hurt at high resolution.
• On Navier-Stokes with nu=1e-3, FFNO-2D error is about 50% higher than S4FFNO-2D at noise levels above 0.128.

Measured effects reported in the paper

Surprising findings

• Multi Input FFNO, which uses explicit past frames, underperforms FFNO at resolution 128, suggesting uncompressed multi-frame input can hurt training.
• Adding small input noise helps training at nu=1e-3 in Navier-Stokes, but not at nu=1e-5.

Limitations

• Observation noise experiments only use i.i.d. Gaussian noise on inputs.
• The theoretical example is a simple linear PDE; the authors note omega_f alone does not determine memory benefit.
• Real-world settings with non-i.i.d. noise or aliasing are not studied.

Experimental setup

Datasets: Kuramoto-Sivashinsky (self-generated) PDEBench Burgers' (nu=0.001) Navier-Stokes 2D (generated following Li et al. FNO) PDE-Refiner KS datasets

Benchmarks: Kuramoto-Sivashinsky (low viscosity, multiple resolutions) Burgers' PDEBench nu=0.001 Navier-Stokes 2D with observation noise

Metrics: nRMSE

Baselines: Factformer Galerkin Transformer U-Net neural operator FFNO Multi Input FFNO LSTM-FFNO

Builds on: Fourier Neural Operator Factorized FNO S4 Mori-Zwanzig formalism PDE-Refiner

Closest papers by meaning

• Hierarchical Implicit Neural Emulators (NeurIPS 2025 · similarity 0.88)
• Poseidon: Efficient Foundation Models for PDEs (NeurIPS 2024 · similarity 0.87)
• CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.87)
• Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data (NeurIPS 2025 · similarity 0.87)
• Stable Long-Horizon PDE Forecasting via Latent Structured Spectral Propagators (NeurIPS 2026 · similarity 0.87)
• Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent Parametric Partial Differential Equations (ICML 2024 · similarity 0.86)