Research map / Vision-language and multimodal / Cross-modal learning

DAMM-Diffusion: Learning Divergence-Aware Multi-Modal Diffusion Model for Nanoparticles Distribution Prediction

CVPR 2025 · Vision-language and multimodal › Cross-modal learning · approach: Medical multimodal fusion · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DAMM-Diffusion, a latent diffusion model with uni-modal (vessel) and multi-modal (vessel+nuclei) branches in a unified U-Net, using a divergence-aware predictor to choose between branch outputs for nanoparticle distribution prediction.

Problem

Predicting nanoparticle distribution in tumors from TME components (vessels and nuclei), where divergence between modalities can make the best uni-modal model outperform joint multi-modal generators.

Approach

A unified diffusion network runs both a uni-modal and a multi-modal branch per reverse step; uncertainty-aware cross-attention (UAFM) estimates a divergence value, and DAMMP selects the multi-modal output only when divergence is below a threshold, trained with a divergence feedback loss.

Key results

• On internal validation, DAMM-Diffusion reaches 96.54% SSIM and 47.93 dB PSNR, versus 94.36% and 45.02 dB for the best baseline CoLa-Diff.
• On external validation (B16 model), DAMM-Diffusion reaches 88.79% SSIM and 41.95 dB PSNR, versus 86.72% and 40.28 dB for CoLa-Diff.
• On BRATS many-to-one synthesis, DAMM-Diffusion's uni-modal branch outperforms baselines across three tasks, e.g., 24.23 dB PSNR for T1,T2→FLAIR versus 23.37 dB for CoLa-Diff.

Measured effects reported in the paper

Surprising findings

• The best uni-modal diffusion models (LDM, BBDM) outperform several multi-modal GAN-based methods such as DiamondGAN and GANDA, which the authors attribute to unmodeled modality divergence.
• Using vessel images rather than nuclei as the uni-modal branch input gives better results (96.54 vs 95.22 SSIM internally).

Limitations

• Evaluated mainly on a single NP dataset from mice models of breast cancer, with the dataset not released.
• Relies on hyperparameters (alpha, lambda, gamma) that require tuning; gamma strongly affects results without DFL.
• Reports only PSNR and SSIM without clinical or biological validation of predicted NP distributions.

Experimental setup

Datasets: NP distribution dataset from Tang et al. (T1 breast cancer mice, vessel/nuclei patches) BRATS

Benchmarks: NP distribution internal validation (six-fold CV) NP distribution external validation (B16 model) BRATS brain MRI synthesis

Metrics: PSNR SSIM

Baselines: Pix2pix CycleGAN LDM BBDM DiamondGAN GANDA HiNet ResViT HRPN Collaborative Diffusion DiffDP CoLa-Diff

Builds on: Latent Diffusion Models BBDM CoLa-Diff DiffDP

Closest papers by meaning

• Physics-Regularized Multi-Modal Image Assimilation for Brain Tumor Localization (NeurIPS 2024 · similarity 0.85)
• Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment (NeurIPS 2024 · similarity 0.84)
• ZoomLDM: Latent Diffusion Model for Multi-scale Image Generation (CVPR 2025 · similarity 0.83)
• Learned Reference-based Diffusion Sampler for multi-modal distributions (ICLR 2025 · similarity 0.83)
• Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance (NeurIPS 2026 · similarity 0.83)
• Leveraging Tumor Heterogeneity: Heterogeneous Graph Representation Learning for Cancer Survival Prediction in Whole Slide Images (NeurIPS 2024 · similarity 0.83)