Research map / Probabilistic and scientific modeling / Differential equations and PDEs

Compute-optimal data scaling for neural surrogates via multi-fidelity training

NeurIPS 2026 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Neural PDE solvers

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that training neural PDE surrogates on a softmax-weighted mixture of simulation fidelities (discretization and solver error) under a fixed data-generation compute budget matches high-fidelity-only accuracy with up to 6.3x less data-generation compute.

Problem

Neural surrogates for PDEs are trained on synthetic simulator data, and data generation compute (not training compute) becomes the bottleneck for scaling; current practice generates all data at one fixed high fidelity.

Approach

Treat simulator fidelity as a fine-grained discrete spectrum with a tunable per-sample cost; sample training fidelities from a softmax distribution parameterized by lambda, condition the surrogate on normalized fidelity, and choose lambda per budget to trade sample count for accuracy along the Pareto frontier.

Key results

• Multi-fidelity training matches high-fidelity-only accuracy with up to 6.3x less data generation compute, mean around 3.1x across 20 setups.
• Fidelity as a fine-grained 20-level spectrum gives 33% lower error than a binary two-fidelity mixture at the same budget (Transolver, hyperelasticity).
• Calibrated extrapolation of optimal mixture lambda* = a + b log(B) yields mean held-out regret under 5% on solver-error setups and up to 10% on discretization setups.

Surprising findings

• Multi-fidelity over solver error yields larger savings than over discretization error, since neural surrogates struggle to generalize across resolutions spanning two orders of magnitude.
• Savings are largest in the low-to-medium budget regime and the high-fidelity and multi-fidelity curves converge at very large budgets.

Limitations

• Assumes data generation cost dominates training cost and that models are large enough to absorb the extra data.
• Extrapolation recipe for optimal mixture degrades on some discretization error settings.
• Only steady-state, moderate-scale problems; modeling error and transient dynamics not studied.

Experimental setup

Datasets: Synthetic Poisson 1D/2D/3D grid data Synthetic linear elasticity FEM meshes Synthetic hyperelasticity FEM meshes Synthetic 2D RANS airfoil meshes

Benchmarks: Poisson 1D/2D/3D linear elasticity 2D meshes hyperelasticity 2D meshes airfoil RANS 2D

Metrics: relative L2 error data generation cost savings ratio regret of extrapolated optimal mixture

Baselines: High-fidelity only training Binary two-fidelity mixture FNO U-Net Transolver AB-UPT

Builds on: Fourier Neural Operator Transolver AB-UPT Kaplan scaling laws Chinchilla scaling laws

Closest papers by meaning

• Geometric and Physical Constraints Synergistically Enhance Neural PDE Surrogates (ICML 2025 · similarity 0.90)
• Breakeven complexity: A new perspective on neural partial differential equation solvers (NeurIPS 2026 · similarity 0.90)
• Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data (NeurIPS 2025 · similarity 0.89)
• Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling (ICML 2024 · similarity 0.88)
• Active Learning for Neural PDE Solvers (ICLR 2025 · similarity 0.88)
• Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs (ICLR 2024 · similarity 0.87)