Research map / Vision-language and multimodal / Cross-modal learning

Incomplete Modality Disentangled Representation for Ophthalmic Disease Grading and Diagnosis

AAAI 2025 · Vision-language and multimodal › Cross-modal learning · approach: Medical multimodal fusion · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IMDR, a teacher-student distillation framework that disentangles modality-shared and modality-specific features via mutual-information minimization guided by a product-of-experts joint distribution, plus a joint proxy learning module, for ophthalmic disease grading under missing modalities.

Problem

Diagnosing ophthalmic diseases from fundus and OCT images when one modality is missing (inter-modality missing or intra-modality noise), where existing latent-subspace methods learn implicit representations that lose modality-specific information and suffer from modality heterogeneity and redundancy

Approach

Explicitly disentangle features into modal-shared and modal-specific components by minimizing mutual information, with the shared component sampled from a product-of-Gaussians joint distribution whose experts are refined by class-level learnable proxies (joint proxy learning), and distill this teacher knowledge into a student that handles incomplete inputs.

Key results

• On Harvard-30k AMD with complete modalities (Transformer backbone), IMDR beats Eye-Most by over +3.58% accuracy.
• With OCT missing on Harvard-30k AMD (ResNet-50), IMDR reaches 0.752 ACC and 0.805 AUC in the ablation, versus 0.654 ACC for the C-Dist distillation baseline.
• On GAMMA with complete modalities (ResNet-50), IMDR achieves 0.8700 ACC versus 0.8600 for Eye-Most.

Measured effects reported in the paper

Surprising findings

• Under inter-modality missing, IMDR with fundus-only input (0.7517 ACC on AMD) outperforms several complete-modality baselines such as HFS-IL and B-EF.

Limitations

• Gains on some settings are modest or inconsistent (e.g., GAMMA AUC below Eye-Most in complete-modality ResNet-50 setting).
• Relies on a teacher trained with complete modalities and a co-trained student, adding training complexity.
• Hyperparameter sensitivity (proxy ratio r and temperature tau) is reported only on one dataset.

Experimental setup

Datasets: GAMMA Harvard-30k AMD Harvard-30k DR Harvard-30k Glaucoma

Benchmarks: GAMMA Harvard-30k AMD Harvard-30k DR Harvard-30k Glaucoma

Metrics: ACC AUC F1 Specificity

Baselines: B-IF B-EF HFS-IL CR-AF M2LC Eye-Most PE PCME

Builds on: CLUB product-of-experts probabilistic embeddings PCME reparameterization trick

Closest papers by meaning

• An Information Criterion for Controlled Disentanglement of Multimodal Data (ICLR 2025 · similarity 0.87)
• Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts (NeurIPS 2024 · similarity 0.85)
• DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency (AAAI 2024 · similarity 0.85)
• Incongruent Multimodal Federated Learning for Medical Vision and Language-based Multi-label Disease Detection (AAAI 2025 · similarity 0.84)
• RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities (NeurIPS 2026 · similarity 0.83)
• Knowledge Bridger: Towards Training-Free Missing Modality Completion (CVPR 2025 · similarity 0.83)