Research map / Probabilistic and scientific modeling / Variational and generative sampling
FMMI: Flow Matching Mutual Information Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FMMI, a flow-matching estimator of mutual information and O-information that learns a velocity field transforming the product of marginals into the joint distribution (or marginal to conditional) and reads MI from the expected divergence.
Problem
Estimating mutual information from finite samples in high dimensions and at high MI values is hard; discriminative classifier-based estimators are limited by GEBM-style suboptimality, and generative estimators approximate all densities from scratch.
Approach
Instead of training a classifier to separate joint from marginal samples, learn a continuous normalizing flow via flow matching that transforms the product of marginals into the joint (or marginal into conditional); MI equals the negative expected divergence of the learned velocity field, estimated via Hutchinson trace estimation. A time-amortized variant (FMOI) extends this to O-information.
Key results
- On mutinfo benchmarks, cFMMI consistently outperforms MINDE and InfoBridge across dimensions and high-MI settings while using about 20x fewer gradient steps.
- FMOI recovers Gaussian O-information within roughly 10-15% relative error for dimensions 3 to 8 using 10^3 samples.
- FMOI on HCP fMRI gives near-zero O-information for brain states with few significant regions and consistently high positive O-information for states involving many regions.
Limitations
- Remains relatively demanding in computational and sample complexity.
- Theoretical guarantees rely on smoothness and compact support assumptions and Lipschitz log-density conditions.
- O-information on fMRI reported as box plots without ground truth; validation limited to synthetic Gaussian case for O-information.
Experimental setup
Datasets: mutinfo benchmark distributions (Correlated Normal, Half-cube Correlated Normal, Correlated Uniform, Smoothed Uniform) synthetic Wishart-based Gaussian covariance matrices HCP 1200 Subjects fMRI data (preprocessed by Kirova et al., 2025)
Benchmarks: mutinfo distributions (Kholkin et al., 2026 protocol) synthetic Gaussian O-information HCP fMRI O-information
Metrics: MI estimate vs ground truth across dimensions absolute error of O-information relative error
Baselines: InfoBridge MINDE-C MINE InfoNCE RFMI
Builds on: flow matching RFMI DRE-infinity MINDE InfoBridge
Closest papers by meaning
- Mutual Information Estimation via Normalizing Flows (NeurIPS 2024 · similarity 0.90)
- Mutual Information Estimation via $f$-Divergence and Data Derangements (NeurIPS 2024 · similarity 0.87)
- InfoNet: Neural Estimation of Mutual Information without Test-Time Optimization (ICML 2024 · similarity 0.86)
- Stream-level Flow Matching with Gaussian Processes (ICML 2025 · similarity 0.85)
- A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets (NeurIPS 2024 · similarity 0.85)
- Neural Mutual Information Estimation with Vector Copulas (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.