Research map / Probabilistic and scientific modeling / Molecules and proteins

Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment

NeurIPS 2025 · Probabilistic and scientific modeling › Molecules and proteins · approach: Statistical analysis of ML practice · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DiPro, a framework that disentangles static anatomy from dynamic pathology in sequential chest X-rays and fuses them hierarchically with asynchronous EHR data for disease progression identification and ICU prediction on MIMIC.

Problem

Modeling disease progression from longitudinal multimodal data is hampered by redundancy in consecutive chest X-rays (static anatomy dominates) and temporal misalignment between sparse, irregular CXRs and continuous EHR time series.

Approach

Region-level static/dynamic disentanglement of consecutive CXR pairs with orthogonality, static-temporal consistency, and reversed-pair progression-aware losses, followed by multiscale fusion: interval-level local cross-attention aligning EHR with each CXR pair and sequence-level global fusion.

Key results

• On disease progression identification, DiPro reaches macro F1 0.466 ± 0.018 in the multimodal setting versus 0.449 for the best baseline UTDE, and F1 0.453 unimodal versus 0.393 for SDPL.
• On in-hospital mortality, DiPro with longitudinal CXR and EHR achieves AUPRC 0.742 ± 0.003 and AUROC 0.897 ± 0.002, versus 0.722 for UMSE.
• On length-of-stay, DiPro achieves kappa 0.248 ± 0.008 and accuracy 0.440 ± 0.007, versus 0.217 kappa for UMSE.

Measured effects reported in the paper

Surprising findings

• Naively using longitudinal CXRs does not consistently help baselines: UMSE and DrFuse drop on some ICU tasks when moving from last CXR to longitudinal CXRs.
• Adding the STD module to a simple encode-concatenate-attention baseline degraded ICU performance (A3 vs A4), suggesting disentanglement alone is insufficient.

Limitations

• Relies on Chest ImaGenome anatomical bounding-box annotations; automated RGRG boxes cause a slight performance drop.
• Excludes ICU stays with only a single CXR, which may introduce sampling bias and reduce cohort size.
• No explicit longitudinal image registration; relies on bounding-box cropping for alignment.

Experimental setup

Datasets: MIMIC-IV MIMIC-CXR Chest ImaGenome

Benchmarks: MIMIC-IV MIMIC-CXR Chest ImaGenome

Metrics: macro precision macro recall macro F1 AUPRC AUROC accuracy Cohen's kappa

Baselines: CheXRelNet CheXRelFormer SDPL UTDE UMSE MedFuse DrFuse Transformer Gemma3 Qwen2.5-VL Med-Flamingo

Builds on: ResNet-50 Transformer CheXRelNet DrFuse MedFuse

Closest papers by meaning

• Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation (NeurIPS 2024 · similarity 0.89)
• Unscrambling disease progression at scale: fast inference of event permutations with optimal transport (NeurIPS 2024 · similarity 0.87)
• Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generation (CVPR 2025 · similarity 0.86)
• Time-to-Event Pretraining for 3D Medical Imaging (ICLR 2025 · similarity 0.86)
• HC-LLM: Historical-Constrained Large Language Models for Radiology Report Generation (AAAI 2025 · similarity 0.85)
• Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction (NeurIPS 2026 · similarity 0.85)