Research map / Learning theory and algorithms / Kernels and regression theory

Measure-to-measure Regression with Transformers

NeurIPS 2026 · Learning theory and algorithms › Kernels and regression theory · approach: Kernel regression under shift · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes static and dynamic transformer-based methods for measure-to-measure (M2M) regression, with M2M-TFM (transformer flow matching velocity fields) as the main approach, evaluated on synthetic shapes, McKean-Vlasov systems, and patient-derived organoid cancer response data.

Problem

Learning nonlinear operators between probability measures (point clouds) from observed input-output measure pairs, with generalization to unseen measures.

Approach

Use a transformer as a measure-dependent velocity field inside flow matching (M2M-TFM), so the transformer's mean-field attention provides distribution dependence along the whole trajectory without a separate distribution encoder; also a one-step static transformer map trained with distributional losses.

Key results

• On multi-measure objects, M2M-TFM reaches W1 0.1300 and ED 0.0023 (diffusion) versus best baseline CFM W1 0.2306 / ED 0.0071 type gaps, with the lowest variance.
• On 100-dimensional Kuramoto, M2M-TFM achieves W1 3.229 vs 7.303 for Meta-FM and 11.312 for CFM.
• On the PDO patient dataset, M2M-TFM attains the best ED (0.4715) and r^2 (0.9029), with W1 4.3526 close to the best static M2M-W1 (4.1005).

Surprising findings

• Static methods trained on a metric often excel on that metric but underperform on counterpart metrics in the patient data.
• WFM, designed for distribution alignment, performs poorly when applied to M2M regression.

Limitations

• Only linear interpolation paths considered; cannot model nonlinear, stochastic, or birth-death dynamics.
• No theoretical guarantees for learning M2M operators via vector-field models.
• Patient dataset small (10 patients), so generalization to unseen patients is difficult.

Experimental setup

Datasets: synthetic letter silhouettes (240 measure pairs) McKean-Vlasov simulated systems (Kuramoto, FitzHugh-Nagumo, Mean-Field Atlas) patient-derived organoid mass cytometry dataset (PDO-21, PDO-27, PDO-75)

Benchmarks: synthetic letter silhouettes McKean-Vlasov systems (2, 50, 100 dims) PDO colorectal cancer organoid dataset

Metrics: W1 energy distance r^2

Baselines: CFM Meta-FM WFM NMKV MF-Transformer M2M-MMD M2M-ED M2M-W1 M2M-OTMSE

Builds on: conditional flow matching Meta Flow Matching Wasserstein Flow Matching mini-batch optimal transport Neural McKean-Vlasov process

Closest papers by meaning

• Transformers can optimally learn regression mixture models (ICLR 2024 · similarity 0.84)
• Stochastic Interpolants with Data-Dependent Couplings (ICML 2024 · similarity 0.84)
• Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold (ICLR 2025 · similarity 0.83)
• Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-based Decoding (NeurIPS 2025 · similarity 0.83)
• Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.83)
• TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.82)