Research map / Vision-language and multimodal / Cross-modal learning

KMD: Koopman Multi-modality Decomposition for Generalized Brain Tumor Segmentation under Incomplete Modalities

CVPR 2025 · Vision-language and multimodal › Cross-modal learning · approach: Medical multimodal fusion · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes KMD, a plug-and-play Koopman-invariant-subspace module that decomposes modality-shared and modality-specific features to improve brain tumor segmentation under balanced and imbalanced missing MRI modalities, integrated into RFNet, mmFormer, M2FTrans and ShaSpec.

Problem

Brain tumor segmentation from multi-modal MRI when modalities (T1, T2, T1ce, Flair) are missing, especially with imbalanced missing rates across modalities, where shared-latent-space methods become biased toward frequently available modalities.

Approach

Use a Koopman invariant subspace to model bi-directional linear mappings between modalities, with losses that align modality-shared features, enforce orthogonality of modality-specific features, and enforce Koopman-consistency, so the module can be plugged into existing segmentation backbones.

Key results

• On BraTS2018 balanced missing rates, mmFormer average Dice rises from 71.12 to 78.26 with KMD losses (ablation table), and ET Dice improves from 57.89 to 72.29.
• On BraTS2020 imbalanced missing rates, M2FTrans with KMD reaches average Dice around 74.6 across missing-rate settings versus about 64.2 for the baseline.
• Ablation shows the Koopman loss Lk gives the largest ET gain, and adding Lc and Lo yields the best average (RFNet 74.41, mmFormer 78.26, M2FTrans 79.19 total Dice averaged over WT/TC/ET).

Measured effects reported in the paper

Surprising findings

• Passion alone yields limited gains on TC and ET, and sometimes produces false detections despite higher Dice.

Limitations

• Relies on heuristic loss weighting (alpha, beta, gamma) tuned on the dataset.
• Theoretical propositions are informal and assume that modalities span a Koopman invariant subspace without verification.
• Evaluated only on two brain MRI datasets (BraTS2018 and BraTS2020).

Experimental setup

Datasets: BraTS2018 BraTS2020

Benchmarks: BraTS2018 BraTS2020

Metrics: Dice score

Baselines: RFNet mmFormer M2FTrans Passion ShaSpec D2-Net Correlation-Fusion

Builds on: Koopman invariant subspace Passion RFNet mmFormer M2FTrans ShaSpec

Closest papers by meaning

• Incomplete Multi-modal Brain Tumor Segmentation via Learnable Sorting State Space Model (CVPR 2025 · similarity 0.87)
• Semantic-guided Masked Mutual Learning for Multi-modal Brain Tumor Segmentation with Arbitrary Missing Modalities (AAAI 2025 · similarity 0.86)
• UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation (NeurIPS 2025 · similarity 0.86)
• TMFormer: Token Merging Transformer for Brain Tumor Segmentation with Missing Modalities (AAAI 2024 · similarity 0.85)
• Federated Modality-Specific Encoders and Multimodal Anchors for Personalized Brain Tumor Segmentation (AAAI 2024 · similarity 0.84)
• Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention (CVPR 2024 · similarity 0.84)