Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent Parametric Partial Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Vectorized Conditional Neural Fields (VCNeFs), a linear-transformer-conditioned neural field that predicts time-continuous PDE solutions for multiple spatio-temporal query points in parallel, conditioned on initial conditions and PDE parameters.
Problem
Neural surrogate models for time-dependent parametric PDEs lack generalization to unseen PDE parameters, zero-shot super-resolution, continuous temporal extrapolation, dimension-agnostic support, and efficient long rollouts; Transformer-based PDE solvers also suffer from quadratic cost.
Approach
Represent PDE solutions as a conditional neural field whose spatio-temporal query coordinates are encoded as a set, processed with linear self-attention to capture spatial dependencies, and modulated (FiLM-style scaling) by a latent encoding of the initial condition and PDE parameters, so all queried times and points are predicted in one forward pass.
Key results
- On 1D Burgers, VCNeF-R reaches nRMSE 0.0784 vs. 0.0987 for FNO and 0.0566 for UNet (UNet best on nRMSE), while VCNeF-R has the best bRMSE of 0.0179 among compared models.
- On 2D CNS (64x64), VCNeF reaches nRMSE 0.1994 vs. 0.5625 for FNO, a 64.6% reduction.
- For temporal zero-shot super-resolution on 1D CNS, VCNeF's nRMSE changes only from 0.2943 (41 steps) to 0.2965 (82 steps), while CORAL degrades from 0.5993 to 1.1524.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add randomized starting points during training | VCNeF · Burgers | nRMSE: 0.0824 -> 0.0784 | Helped |
| add randomized starting points during training | VCNeF · Advection | bRMSE: 0.0088 -> 0.0040 | Helped |
| add randomized starting points during training | VCNeF · 1D CNS | nRMSE: 0.2943 -> 0.2029 | Helped |
| add randomized starting points during training | VCNeF · 1D CNS | bRMSE: 1.3496 -> 1.1366 | Hurt |
| remove spatial self-attention (ablation) | VCNeF · 1D Burgers | nRMSE: 0.0824 -> 0.8890 | Hurt |
| remove spatial self-attention (ablation) | VCNeF · 1D Burgers | bRMSE: 0.0228 -> 0.3242 | Hurt |
| add FiLM shifting to scaling (ablation) | VCNeF · 1D Burgers | nRMSE: 0.0824 -> 0.0945 | Hurt |
| add FiLM shifting to scaling (ablation) | VCNeF · 1D Burgers | bRMSE: 0.0228 -> 0.0291 | Hurt |
| use only large patches instead of small and large patches (ablation) | VCNeF · 2D CNS | nRMSE: 0.1994 -> 0.4569 | Hurt |
| use only large patches instead of small and large patches (ablation) | VCNeF · 2D CNS | bRMSE: 0.0904 -> 0.1982 | Hurt |
| replace linear attention with vanilla attention (ablation) | VCNeF · 1D Burgers, 256 spatial resolution | GPU memory: 31.4 GiB -> 72.6 GiB | Hurt |
| replace linear attention with vanilla attention (ablation) | VCNeF · 1D Burgers, 512 spatial resolution | time per epoch: 32 s -> 78 s | Hurt |
Surprising findings
- On 1D Advection and Burgers, the UNet baseline achieves lower nRMSE than VCNeF, so VCNeF is not uniformly best.
- Replacing linear attention with vanilla attention on 1D Burgers required over 640 GiB at 1024 tokens, illustrating memory costs of quadratic attention.
Limitations
- Parallel multi-timestep prediction requires GPUs with large memory; sequential mode trades memory for inference time.
- Patch-based encoders restrict 2D/3D use to regular grids.
- Evaluated only on PDEBench hydrodynamic datasets; no turbulent simulations.
Experimental setup
Datasets: PDEBench 1D Burgers PDEBench 1D Advection PDEBench 1D CNS PDEBench 2D CNS PDEBench 3D CNS
Benchmarks: PDEBench 1D Burgers PDEBench 1D Advection PDEBench 1D CNS PDEBench 2D CNS PDEBench 3D CNS
Metrics: nRMSE bRMSE inference time GPU memory
Baselines: FNO cFNO MP-PDE UNet (PDEArena) CORAL Galerkin Transformer OFormer
Builds on: Conditional Neural Fields Linear Transformer FiLM Fourier Neural Operator CORAL
Closest papers by meaning
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.90)
- Space-Time Continuous PDE Forecasting using Equivariant Neural Fields (NeurIPS 2024 · similarity 0.89)
- Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers (ICML 2025 · similarity 0.89)
- Breakeven complexity: A new perspective on neural partial differential equation solvers (NeurIPS 2026 · similarity 0.88)
- Active Learning for Neural PDE Solvers (ICLR 2025 · similarity 0.88)
- AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.