Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Spectral Convolutional Conditional Neural Process
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SConvCNP, a convolutional neural process whose decoder uses FNO-style global spectral convolutions in the frequency domain instead of local CNN kernels, and evaluates it on synthetic 1-D, Lotka-Volterra, traffic-flow and image-completion regression.
Problem
Convolutional conditional neural processes (ConvCNPs) use local kernels with small receptive fields, so they struggle to aggregate information from sparse observations across large domains; enlarging kernels is costly, and transformer NPs scale quadratically.
Approach
Replace the ConvCNP's U-Net decoder with a U-shaped Fourier neural operator that parameterizes convolution kernels directly over a finite set of Fourier modes, giving global receptive fields at near-linear cost in grid size.
Key results
- On synthetic sawtooth-wave tasks SConvCNP reaches log-likelihood 0.82 vs -0.90 for CNP/AttCNP/TNP/TE-TNP, where TNP and TE-TNP collapsed to zero predictions.
- On California traffic flow SConvCNP achieves the best log-likelihood of 2.05 versus 1.98 for ConvCNP and 1.93 for TNP.
- On Lotka-Volterra SConvCNP matches ConvCNP on log-likelihood (0.14) and is close to TE-TNP (0.16), with RMSE of 0.30 on par with TE-TNP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the ConvCNP U-Net backbone with a U-shaped FNO backbone | ConvCNP · Sawtooth | log-likelihood: 0.10_{\pm0.30} -> 0.82_{\pm0.03} | Helped |
| Replace the ConvCNP U-Net backbone with a U-shaped FNO backbone | ConvCNP · Sawtooth | RMSE: 0.40_{\pm0.06} -> 0.24_{\pm0.00} | Helped |
| Replace the ConvCNP U-Net backbone with a U-shaped FNO backbone | ConvCNP · Periodic | log-likelihood: -0.73_{\pm0.01} -> -0.66_{\pm0.00} | Hurt |
| Replace the ConvCNP U-Net backbone with a U-shaped FNO backbone | ConvCNP · California traffic flow | log-likelihood: 1.98_{\pm0.02} -> 2.05_{\pm0.02} | Helped |
| Add positional encodings to the functional embedding (ablation) | SConvCNP · Experiments (setting not specified) | predictive accuracy | Helped |
Surprising findings
- TNP and TE-TNP failed to train on sawtooth-wave tasks, with predictions collapsing to zero regardless of capacity or initialization, which the authors attribute to spectral bias.
- The DFT-based model is sensitive to grid resolution and physical extent, so fixed domain and resolution must be controlled for generalization.
Limitations
- Grid size grows exponentially with input dimension, restricting the method to low-dimensional domains.
- DFT-based spectral convolution is sensitive to grid resolution and physical range, requiring fixed domain choices.
- Positional encodings break translation equivariance.
Experimental setup
Datasets: GP Matern-5/2 synthetic GP periodic synthetic sawtooth synthetic square-wave synthetic stochastic Lotka-Volterra simulations LargeST California traffic flow DTD
Benchmarks: Matern-5/2 GP periodic GP sawtooth square wave Lotka-Volterra LargeST California traffic flow DTD image completion
Metrics: log-likelihood RMSE
Baselines: CNP AttCNP TNP TE-TNP ConvCNP
Builds on: ConvCNP Fourier neural operator conditional neural process TE-TNP
Closest papers by meaning
- Spectral Transformer Neural Processes (NeurIPS 2026 · similarity 0.87)
- Flow Matching Neural Processes (NeurIPS 2025 · similarity 0.86)
- Neural Operators with Localized Integral and Differential Kernels (ICML 2024 · similarity 0.85)
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.84)
- Sequence Modeling with Spectral Mean Flows (NeurIPS 2025 · similarity 0.84)
- Deep Kernel Posterior Learning under Infinite Variance Prior Weights (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.