Research map / Probabilistic and scientific modeling / Molecules and proteins
Modeling Microenvironment Trajectories on Spatial Transcriptomics with NicheFlow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NicheFlow, a flow-matching model with optimal transport and factorized variational flow matching that predicts the evolution of cellular microenvironments as point clouds across spatial transcriptomics time points.
Problem
Modeling the temporal evolution of tissue microenvironments (niches) from time-resolved spatial transcriptomics, where cells cannot be matched across destructive slides and existing trajectory methods operate at single-cell level.
Approach
Represent local radius-based cell neighborhoods as attributed point clouds and learn a source-conditioned flow between niches using entropic OT mini-batch coupling on pooled coordinate/feature representations, trained with a mixed-factorized variational flow matching loss (Laplace for coordinates, Gaussian for expression).
Key results
- NicheFlow with GLVFM reaches 1NN-F1 0.664 on mouse embryonic development versus 0.540 for LUNA and 0.586 for the best RPCFlow variant, with the lowest PSD (0.883) and SPD (0.398).
- On axolotl brain development, NicheFlow GLVFM achieves 1NN-F1 0.628 versus 0.524 for the best RPCFlow CFM variant and 0.331 for LUNA.
- On mouse brain aging, NicheFlow is competitive rather than best; RPCFlow CFM has the best PSD (1.543) and NicheFlow CFM/GVFM are best on SPD (0.556/0.531 range), with 1NN-F1 at most 0.285 versus LUNA 0.222.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GLVFM instead of CFM for NicheFlow (ablation) | NicheFlow · MED | 1NN-F1: 0.609 -> 0.664 | Helped |
| Use GLVFM instead of GVFM for NicheFlow (ablation) | NicheFlow · MED | 1NN-F1: 0.596 -> 0.664 | Helped |
| Use GLVFM instead of CFM for NicheFlow (ablation) | NicheFlow · ABD | 1NN-F1: 0.604 -> 0.628 | Helped |
| Use GLVFM instead of GVFM for NicheFlow (ablation) | NicheFlow · ABD | 1NN-F1: 0.574 -> 0.628 | Helped |
| Use GLVFM instead of CFM for NicheFlow (ablation) | NicheFlow · MBA | 1NN-F1: 0.283 -> 0.285 | Helped |
| Use GLVFM instead of GVFM for NicheFlow (ablation) | NicheFlow · MBA | 1NN-F1: 0.268 -> 0.285 | Helped |
| Model spatial microenvironments rather than independently modeling single cells | NicheFlow versus SPFlow · MED | SPD: 0.602 -> 0.398 | Helped |
| Use spatially co-localized neighborhoods rather than randomly sampled point clouds | NicheFlow versus RPCFlow · MED | SPD: 0.586 -> 0.398 | Helped |
| Increase the moscot spatial-weight parameter to 0.75 from 0.5 (ablation) | moscot spinal cord mapping · Spinal Cord | proportion mapped to correct cell type: 0.712 -> 0.729 | Helped |
| Increase the moscot spatial-weight parameter to 0.9 from 0.5 (ablation) | moscot neural crest mapping · Neural Crests | proportion mapped to correct cell type: 0.162 -> 0.112 | Hurt |
Surprising findings
- On the mouse brain aging dataset, the proposed model does not uniformly outperform the random point cloud baseline, with RPCFlow CFM achieving the best PSD.
Limitations
- Fixed OT feature weighting lambda during training limits flexibility at inference.
- Radius-based niche definition may be suboptimal for small or irregular microenvironments; spatial slides must be pre-aligned and normalized.
- Evaluated only in-distribution; does not generalize to unseen slides or regions and does not model division or death.
Experimental setup
Datasets: mouse embryonic development Stereo-seq (E9.5-E11.5) axolotl brain development Stereo-seq (five time points) mouse brain aging MERFISH (twenty time points)
Benchmarks: mouse embryonic development axolotl brain development mouse brain aging
Metrics: 1NN-F1 point-to-shape distance (PSD) shape-to-point distance (SPD)
Baselines: SPFlow RPCFlow LUNA moscot CFM
Builds on: flow matching variational flow matching generative entropic optimal transport flow matching moscot LUNA
Closest papers by meaning
- Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching (ICML 2025 · similarity 0.87)
- GeneFlow: Translation of Single-cell Gene Expression to Histopathological Images via Rectified Flow (NeurIPS 2025 · similarity 0.87)
- TracingFlow: A Simulation-Free Trajectory Inference Framework Based on Second-Order Dynamics (NeurIPS 2026 · similarity 0.86)
- Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images (ICLR 2025 · similarity 0.85)
- Global Context-aware Representation Learning for Spatially Resolved Transcriptomics (ICML 2025 · similarity 0.84)
- SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.