Research map / Vision-language and multimodal / Cross-modal learning
MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MedM2G, a unified latent diffusion framework that aligns text, CT, MRI and X-ray via central alignment, preserves medical visual invariants with Barlow-Twins-style loss, and links modality diffusers with cross-guided attention for multi-flow medical generation.
Problem
Existing medical generative models are single-flow or two-modality specific, and unified multi-modal generative models fail to capture medical-specific knowledge and lack well-paired cross-modal medical data (e.g., X-ray–MRI).
Approach
Align all modalities through text as a central modality (O(n) pairwise contrastive alignment), preserve per-modality clinical structure with a negative-free Barlow Twins visual invariant loss, and condition each modality's latent diffusion UNet on another modality's context encoding plus a learnable textual-inversion-style adaptation via shared cross-attention, trained in three paired rounds.
Key results
- On MIMIC-CXR report generation, BLEU-4 reaches 0.142 vs. 0.124 for the best listed baseline (METransformer/COMG range), and on IU X-Ray BLEU-1 is 0.533.
- On chest X-ray generation (MIMIC-CXR), FID drops to 1.7 vs. 2.5 for UniXGen and 52.7 for original SD.
- On Pelvic MRI-CT translation, MedM2G reaches 27.45 PSNR for T2→CT, above SynDiff's 26.86.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MedM2G instead of METransformer | Medical report generation · MIMIC-CXR | ROUGE-L: 0.291 -> 0.309±0.009 | Helped |
| Use MedM2G instead of CoLa-Diff | MRI synthesis · BraTS, T2+T1ce+FLAIR→T1 | PSNR: 28.26±3.13 -> 29.89±2.26 | Helped |
| Use MedM2G instead of UniXGen | Chest X-ray generation · MIMIC-CXR | FID: 2.5 -> 1.7 | Helped |
| Use MedM2G instead of SynDiff | MRI-CT translation · Pelvic, T2→CT | PSNR: 26.86±0.51 -> 27.45±0.64 | Helped |
| Add central alignment strategy (ablation) | MedM2G · MIMIC-CXR test set | ROUGE-L: 0.261±0.013 -> 0.274±0.012 | Helped |
| Add latent cross-guided alignment generation (ablation) | MedM2G · ACDC | FID: 8.3 -> 4.3 | Helped |
| Add medical visual invariant preservation (ablation) | MedM2G · BraTS2020, T2+T1→PD | PSNR: 31.02±2.46 -> 33.12±3.45 | Helped |
| Add all three components: central alignment, latent cross-guided alignment generation, and visual invariant preservation (ablation) | MedM2G · MIMIC-CXR test set | ROUGE-L: 0.261±0.013 -> 0.309±0.009 | Helped |
| Add all three components: central alignment, latent cross-guided alignment generation, and visual invariant preservation (ablation) | MedM2G · ACDC | FID: 8.3 -> 1.7 | Helped |
| Add all three components: central alignment, latent cross-guided alignment generation, and visual invariant preservation (ablation) | MedM2G · Pelvis, T2→CT | PSNR: 24.56±0.78 -> 27.45±0.19 | Helped |
Limitations
- Generated medical content could be misused to fabricate false medical information.
- Generated modalities are only auxiliary for diseases without clinically multimodal features and need comprehensive analysis.
- Paper does not report results on some tasks in tables (several table references unresolved).
Experimental setup
Datasets: MIMIC-CXR MedICat Brain tumor MRI and CT scan BraTS 2020 IXI Gold Atlas male pelvis Chest X-ray IU X-ray SLIVER07 ACDC Chest X-ray14 OpenI
Benchmarks: IU X-Ray MIMIC-CXR BraTS 2020 IXI Gold Atlas Pelvis ChestX-ray14 ACDC SLIVER07 OpenI
Metrics: BLEU-1 BLEU-4 ROUGE-L PSNR SSIM FID NIQE MS-SSIM
Baselines: R2Gen R2GenCMN KiUT METransformer CoLa-Diff LDM SynDiff UniXGen GLIGEN RoentGen
Builds on: Latent Diffusion Models Textual Inversion Barlow Twins Versatile Diffusion CoLa-Diff
Closest papers by meaning
- Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces (ICML 2025 · similarity 0.82)
- Advancing Medical Multimodal Learning and Data Generation with Diffusion Model and LLM (AAAI 2025 · similarity 0.82)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.81)
- Diffusion Models for Multi-Task Generative Modeling (ICLR 2024 · similarity 0.81)
- DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.81)
- MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.