Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that neural emulators trained purely on low-fidelity solver data can beat that solver against a high-fidelity reference, proving it for linear advection/diffusion/Poisson via Fourier analysis and demonstrating it empirically for nonlinear emulators on advection and Burgers.
Problem
Neural emulators of PDEs trained on numerical solver data are assumed to be bounded by the training solver's fidelity; the paper asks whether they can exceed it when evaluated against a higher-fidelity reference.
Approach
Defines a superiority ratio comparing emulator rollout error to training-solver error against a better reference, and shows via closed-form Fourier analysis that simple linear emulators fitted at one mode can beat implicit/explicit upwind solvers (forward, backward, and state-space superiority), with nonlinear UNet/ConvNet/FNO/Transformer emulators showing autoregressive superiority.
Key results
- For linear advection, a two-parameter linear emulator trained on an implicit upwind scheme can beat that scheme at modes above the training mode (superiority ratio below 1), with forward superiority in both magnitude and phase error.
- For the Poisson equation, the emulator shows backward superiority, outperforming an unconverged Richardson iterative solver mainly for test modes below the training mode.
- On the advection equation with gamma_1=-0.9, nearly all tested nonlinear architectures achieve autoregressive superiority (ratio below 1) after some initial steps, though the one-step ratio is at least 1 as expected.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train a low-capacity linear emulator on implicit first-order upwind data and test at higher modes | Linear advection emulator · Advection, higher Fourier modes | superiority ratio | Helped |
| Train a linear emulator on implicit diffusion-scheme data and test at higher modes | Linear diffusion emulator · Diffusion, higher Fourier modes | superiority ratio | Helped |
| Train an emulator on truncated Richardson-iteration outputs and test at lower modes | Poisson emulator · Poisson, lower Fourier modes | superiority ratio | Helped |
| Use nonlinear neural emulators for multi-step rollouts instead of the training solver | Advection neural emulators · Advection, same initial-condition distribution | superiority ratio | Helped |
| Train on implicit-solver data and test on initial conditions with broader mode content | ConvNet · Advection, broader initial-condition distribution | superiority ratio | Helped |
| Use a global-receptive-field architecture rather than a local ConvNet for single-mode to multi-mode extrapolation | FNOs and Transformers · Advection, broader initial-condition distribution | state-space generalization | Hurt |
| Train a UNet on smooth initial snapshots from a solver with a truncated nonlinear solve | Burgers UNet · Burgers, shock propagation | accuracy against converged reference | Helped |
| Train on explicit upwind data within its stability limit and evaluate multi-step rollouts | Advection neural emulators · Advection, same initial-condition distribution | superiority ratio | Helped |
Surprising findings
- Nonlinear emulators show autoregressive superiority even when trained on the relatively accurate explicit upwind scheme, for which superiority could not be proven in the linear case.
- On the advection equation, only the local ConvNet generalizes to multi-mode initial conditions with state-space superiority; FNOs and Transformers do not.
Limitations
- Superiority is regime-dependent and not a universal guarantee; it depends on PDE, solver error, architecture, metric and horizon.
- Theoretical proofs are restricted to linear PDEs with single-mode training and testing; nonlinear results are empirical.
- Superiority is expected to diminish when the training solver is highly accurate.
Experimental setup
Datasets: simulated advection data from implicit and explicit upwind solvers simulated Burgers data from implicit upwind solver with truncated Picard iteration
Benchmarks: linear advection Burgers equation diffusion equation Poisson equation
Metrics: superiority ratio rollout error magnitude error phase error
Baselines: implicit first-order upwind explicit first-order upwind analytical time stepper Richardson iteration solver
Builds on: neural operators unrolled training
Closest papers by meaning
- APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs (NeurIPS 2024 · similarity 0.89)
- Hierarchical Implicit Neural Emulators (NeurIPS 2025 · similarity 0.89)
- Compute-optimal data scaling for neural surrogates via multi-fidelity training (NeurIPS 2026 · similarity 0.89)
- Breakeven complexity: A new perspective on neural partial differential equation solvers (NeurIPS 2026 · similarity 0.89)
- Active Learning for Neural PDE Solvers (ICLR 2025 · similarity 0.87)
- INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.