Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Latent Neural Operator for Solving Forward and Inverse PDE Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Latent Neural Operator (LNO), which encodes PDE samples into a learnable latent space via Physics-Cross-Attention (PhCA), learns the operator there with Transformer self-attention, and decodes at arbitrary query positions for forward and inverse PDE problems.
Problem
Neural operators for PDEs operate on large numbers of sample points in geometric space, which is computationally expensive; inverse problems additionally require predicting values at positions unseen in the observations.
Approach
Learn a latent space from data via cross-attention encoder/decoder (PhCA) that decouples key/query positions from values, so the operator runs on a small set of latent tokens and can decode at any query location; encoder and decoder share an attention projector, and only the first and last layers transform data between spaces.
Key results
- LNO achieves best relative L2 on Darcy (0.49e-2), NS2d (8.45e-2), Elasticity (0.52e-2) and Pipe (0.26e-2) versus Transolver's 0.58, 8.79, 0.62, 0.31.
- Compared with Transolver, LNO reduces GPU memory by about 50% and trains about 1.8x faster per epoch on average (e.g., Darcy 38.98s vs 88.68s per epoch).
- On the inverse Burgers' completer task at 20% observation ratio, LNO reaches 0.60% relative MAE versus 1.12% for GNOT and 2.51% for DeepONet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LNO rather than the strongest listed baseline | Forward PDE benchmarks · Darcy, NS2d, Elasticity, Pipe | Relative L2 error: 0.58 -> 0.49; 8.79 -> 8.45; 0.62 -> 0.52; 0.31 -> 0.26 | Helped |
| Use LNO rather than Transolver | Airfoil and Plasticity · Airfoil, Plasticity | Relative L2 error: 0.47 -> 0.51; 0.12 -> 0.29 | Hurt |
| Use LNO rather than GNOT | Burgers inverse completer · Burgers, observation ratios 20%, 10%, 5%, 1%, 0.5% | Relative MAE: 1.12% -> 0.60%; 1.39% -> 0.74%; 1.62% -> 0.77%; 1.63% -> 1.18%; 2.56% -> 2.05% | Helped |
| Replace PhCA with Physics-Attention (ablation) | LNO · Forward PDE benchmarks | Relative L2 error: Darcy 0.49 -> 0.53; NS2d 8.45 -> 22.65; Airfoil 0.51 -> 0.74; Elasticity 0.52 -> 0.88; Plasticity 0.29 -> 0.49; Pipe 0.2 | Hurt |
| Replace scaled dot-product attention with linear or Galerkin-type attention (ablation) | LNO latent operator · Forward PDE benchmarks | Relative L2 error: See attention ablation table | Mixed |
| Use separate encoder and decoder MLPs instead of shared weights (ablation) | LNO PhCA · Darcy, NS2d, Airfoil, Elasticity, Pipe | Relative L2 error: Darcy 0.49 -> 0.54; NS2d 8.45 -> 10.67; Airfoil 0.51 -> 0.53; Elasticity 0.52 -> 1.09; Pipe 0.26 -> 0.36 | Hurt |
| Use separate encoder and decoder MLPs instead of shared weights (ablation) | LNO PhCA · Plasticity | Relative L2 error: 0.29 -> 0.26 | Helped |
| Increase the number of latent sampling positions (ablation) | LNO · Airfoil, Elasticity, Plasticity, Pipe | Relative L2 error | Mixed |
| Increase the number of latent sampling positions (ablation) | LNO · Darcy, NS2d | Relative L2 error | Helped |
| Increase Transformer depth beyond the optimal range (ablation) | LNO · Forward PDE benchmarks | Prediction error | Mixed |
| Increase Transformer token width (ablation) | LNO · Forward PDE benchmarks | Prediction error | Helped |
| Use LNO rather than Transolver | Forward PDE models · Six forward benchmarks | GPU memory: 50% reduction (approximately) | Helped |
Surprising findings
- Deeper LNO (beyond 4 or 8 Transformer blocks) degrades performance, which the authors attribute to optimization difficulty in latent space.
Limitations
- Time-dependent equations decode intermediate states to geometric space for loss computation, so dynamics are not fully computed in latent space.
- No physical prior is imposed on the latent space.
- Airfoil and Plasticity results are close to but not better than the state of the art (Transolver).
Experimental setup
Datasets: Darcy NS2d Airfoil Elasticity Plasticity Pipe Burgers' equation (generated, 128x128 grid)
Benchmarks: Darcy NS2d Airfoil Elasticity Plasticity Pipe Burgers inverse problem
Metrics: relative L2 relative MAE parameter count GPU memory training time per epoch
Baselines: FNO Geo-FNO F-FNO U-FNO LSM Galerkin Transformer OFormer GNOT FactFormer ONO Transolver DeepONet
Builds on: Transolver DeepONet Galerkin Transformer OFormer FNO
Closest papers by meaning
- Physics-Informed Deep Inverse Operator Networks for Solving PDE Inverse Problems (ICLR 2025 · similarity 0.89)
- RIGNO: A Graph-based Framework For Robust And Accurate Operator Learning For PDEs On Arbitrary Domains (NeurIPS 2025 · similarity 0.88)
- Latent Mamba Operator for Partial Differential Equations (ICML 2025 · similarity 0.88)
- Adaptive Coordinate Transforms for Neural Operators (NeurIPS 2026 · similarity 0.87)
- Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning (NeurIPS 2024 · similarity 0.87)
- Reference Neural Operators: Learning the Smooth Dependence of Solutions of PDEs on Geometric Deformations (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.