Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MFRNP, a multi-fidelity neural process that models the residual between the averaged lower-fidelity decoded predictions and the highest-fidelity ground truth, trained with a Residual-ELBO, and evaluated on PDE benchmarks and global climate temperature prediction.
Problem
Multi-fidelity surrogate modeling at high fidelity suffers when decoder parameters are not shared across fidelities and the highest-fidelity data has limited domain coverage (OOD), hurting extrapolation and scalability to high-dimensional outputs.
Approach
Share information across fidelities via decoded outputs rather than only latent encodings: lower-fidelity NP surrogates predict at highest-fidelity inputs, their average is used as a baseline, and a top-fidelity NP models the residual; a Residual-ELBO jointly trains all fidelities so lower-fidelity decoders are optimized for residual modeling.
Key results
- On Heat/Poisson full setting, MFRNP reports nRMSE around 0.004-0.008 vs 0.015-0.107 for best baseline D-MFD on Heat/Poisson 2-5 fidelity tasks (e.g., Heat 2: 0.005 vs 0.015).
- Under OOD setting, MFRNP achieves e.g. Poisson 2 nRMSE 0.017 vs 0.103 for MFHNP; authors report about 90% average improvement over baselines.
- On climate perfect-model tests, MFRNP outperforms other models in 3 of 4 SSP scenarios and stays consistent from 2015 to 2100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| include decoder outputs in cross-fidelity aggregation instead of hierarchical latent-space sharing (ablation) | MFRNP · Heat 3 Full | nRMSE: 0.013±4.46e-4 -> 0.0039±2.94e-4 | Helped |
| include decoder outputs in cross-fidelity aggregation instead of hierarchical latent-space sharing (ablation) | MFRNP · Heat 5 Full | nRMSE: 0.013±2.76e-4 -> 0.0045±2.94e-4 | Helped |
| include decoder outputs in cross-fidelity aggregation instead of hierarchical latent-space sharing (ablation) | MFRNP · Poisson 3 Full | nRMSE: 0.138±6.1e-3 -> 0.0073±5.25e-4 | Helped |
| include decoder outputs in cross-fidelity aggregation instead of hierarchical latent-space sharing (ablation) | MFRNP · Poisson 5 Full | nRMSE: 0.184±1.77e-2 -> 0.0046±1.2e-4 | Helped |
| include decoder outputs in cross-fidelity aggregation instead of hierarchical latent-space sharing (ablation) | MFRNP · Heat 3 OOD | nRMSE: 0.047±2.72e-4 -> 0.004±3.82e-4 | Helped |
| include decoder outputs in cross-fidelity aggregation instead of hierarchical latent-space sharing (ablation) | MFRNP · Heat 5 OOD | nRMSE: 0.055±1.87e-4 -> 0.012±1.05e-2 | Helped |
| include decoder outputs in cross-fidelity aggregation instead of hierarchical latent-space sharing (ablation) | MFRNP · Poisson 3 OOD | nRMSE: 0.128±1.3e-2 -> 0.018±1.27e-3 | Helped |
| use increasing fidelity weights (0.1, 0.2, 0.3, 0.4) instead of uniform averaging (ablation) | MFRNP · Heat5 Full | nRMSE: 0.0045±2.94e-4 -> 0.0042±1.19e-3 | Helped |
| use increasing fidelity weights (0.1, 0.2, 0.3, 0.4) instead of uniform averaging (ablation) | MFRNP · Poisson5 Full | nRMSE: 0.0046±1.2e-4 -> 0.0103±2.02e-3 | Hurt |
| use residual modeling with decoder-aware information sharing rather than the strongest listed baseline | MFRNP · PDE Full tasks | nRMSE | Helped |
| use residual modeling with decoder-aware information sharing rather than the strongest listed baseline | MFRNP · PDE OOD tasks | nRMSE | Helped |
Surprising findings
- Uniform averaging of lower-fidelity predictions outperformed a fidelity-increasing weighted average on Poisson5 (0.0046 vs 0.0103 nRMSE), showing weighting is dataset-dependent.
Limitations
- Ancestral sampling doubles inference time, which may be costly for expensive encoders/decoders such as neural ODEs.
- Uniform averaging of lower-fidelity predictions may lose detailed information.
- Weighted averaging helps on some tasks but hurts on others, requiring dataset-specific tuning.
Experimental setup
Datasets: Heat equation simulations Poisson equation simulations Navier-Stokes fluid simulations ScenarioMIP climate models ERA5 reanalysis
Benchmarks: Heat 2/3/5 fidelity Poisson 2/3/5 fidelity Fluid (Navier-Stokes) ERA5 earth surface temperature perfect-model test
Metrics: nRMSE latitude-weighted nRMSE
Baselines: SF-NP DMF NARGP MFHNP D-MFD NNGPR DeepESD
Builds on: Neural Processes D-MFD MFHNP NARGP DMF
Closest papers by meaning
- Compute-optimal data scaling for neural surrogates via multi-fidelity training (NeurIPS 2026 · similarity 0.88)
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.85)
- Model-Agnostic Knowledge Guided Correction for Improved Neural Surrogate Rollout (ICLR 2025 · similarity 0.84)
- Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networks (ICLR 2024 · similarity 0.84)
- Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution (AAAI 2024 · similarity 0.83)
- Geometric and Physical Constraints Synergistically Enhance Neural PDE Surrogates (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.