Research map / Probabilistic and scientific modeling / Variational and generative sampling
Flow Matching Neural Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FlowNP, a transformer-based neural process trained with conditional flow matching that predicts velocities for all target points jointly, sampled via ODE solvers, and reports state-of-the-art log-likelihoods on 1D GP, image, and ERA5 weather benchmarks.
Problem
Neural processes struggle with underfitting intricate structure (non-autoregressive models) or with slow, point-by-point sampling and constrained early autoregressive factors (autoregressive models); the goal is an NP that is expressive, parallel in sampling, and amortizes conditioning.
Approach
Treat neural process prediction as conditional flow matching: a transformer receives context tokens (clean values at t=1) and noisy target tokens at flow time t, and outputs the velocity for all target points in parallel; conditional samples are drawn by integrating an ODE, with no guidance or replacement needed.
Key results
- On 1D GP benchmarks FlowNP reaches 1.69 log-likelihood on RBF and 1.30 on Matern-5/2 (vs TNP 1.65 and 1.29), and -0.50 on Periodic (vs TNP -0.58).
- On ERA5 weather data FlowNP achieves 12.79 target log-likelihood vs 11.32 for TNP and 6.76 for NDP.
- Sampling 1 EMNIST sample with 748 target points takes 4.6 s for FlowNP vs 10.4 s for NDP and 72.6 s for TNP on an RTX4090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FlowNP rather than TNP | 1D GP benchmarks · RBF | log-likelihood: 1.65 -> 1.69 | Helped |
| use FlowNP rather than TNP | 1D GP benchmarks · Matern-5/2 | log-likelihood: 1.29 -> 1.30 | Helped |
| use FlowNP rather than NDP | 1D GP benchmarks · Periodic | log-likelihood: -0.54 -> -0.50 | Helped |
| use FlowNP rather than NDP | 1D GP benchmarks · Fixed-Noisy RBF | log-likelihood: 0.71 -> 0.71 | No clear effect |
| use FlowNP rather than TNP | 1D GP benchmarks · Fixed-Noisy Matern-5/2 | log-likelihood: 0.30 -> 0.30 | No clear effect |
| use FlowNP rather than strongest baseline | image and weather benchmarks · EMNIST 0-9 | log-likelihood: 2.08 { $\pm$ 0.02} -> 2.50 { $\pm$ 0.01} | Helped |
| use FlowNP rather than strongest baseline | image and weather benchmarks · EMNIST 10-46 | log-likelihood: 1.80 { $\pm$ 0.05} -> 2.42{ $\pm$ 0.01} | Helped |
| use FlowNP rather than strongest baseline | image and weather benchmarks · CelebA | log-likelihood: 4.28 { $\pm$ 0.04} -> 6.37 { $\pm$ 0.01} | Helped |
| use FlowNP rather than strongest baseline | image and weather benchmarks · ERA5 | log-likelihood: 11.32 { $\pm$ 0.09} -> 12.79 { $\pm$ 0.03} | Helped |
| predict flow velocity instead of clean data (ablation) | conditional FlowNP with linear-vp schedule · RBF | log-likelihood: 1.38 -> 0.41 | Hurt |
| predict flow velocity instead of noise (ablation) | conditional FlowNP with linear-vp schedule · EMNIST | log-likelihood: 1.56 -> 0.48 | Hurt |
| use cond-ot instead of linear-vp schedule (ablation) | FlowNP · RBF | log-likelihood: 0.41 -> 1.69 | Helped |
Surprising findings
- The paper finds its non-autoregressive FlowNP can capture sharp, multimodal step-function transitions where autoregressive TNP, constrained to Gaussians per step, produces noisy transitions.
- Training FlowNP only on joint distributions and computing conditionals via the joint/context ratio gives slightly higher likelihood than conditional training.
Limitations
- Iterative ODE sampling and likelihood computation is slower than one-shot prediction; likelihood requires many network evaluations.
- Consistency (marginal and conditional) is not guaranteed by the architecture, only promoted by the training paradigm.
- Likelihood on image data requires adding noise to avoid unbounded continuous likelihoods.
Experimental setup
Datasets: synthetic 1D Gaussian processes (RBF, Matern-5/2, Periodic) EMNIST CelebA ERA5
Benchmarks: 1D GP RBF 1D GP Matern-5/2 1D GP Periodic Fixed-Noisy RBF Fixed-Noisy Matern-5/2 EMNIST CelebA ERA5
Metrics: target log-likelihood wall-clock sampling time
Baselines: CNP NP ANP TNP NDP
Builds on: neural processes flow matching Transformer Neural Process Neural Diffusion Process
Closest papers by meaning
- Flow Matching with Gaussian Process Priors for Probabilistic Time Series Forecasting (ICLR 2025 · similarity 0.89)
- Stream-level Flow Matching with Gaussian Processes (ICML 2025 · similarity 0.88)
- Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions (ICLR 2025 · similarity 0.87)
- Switched Flow Matching: Eliminating Singularities via Switching ODEs (ICML 2024 · similarity 0.87)
- Stochastic Process Learning via Operator Flow Matching (NeurIPS 2025 · similarity 0.87)
- Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modelling (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.