Research map / Vision-language and multimodal / Cross-modal learning

Knowledge Bridger: Towards Training-Free Missing Modality Completion

CVPR 2025 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Knowledge Bridger, a training-free missing modality completion pipeline that uses a large multimodal model to build knowledge graphs from available modalities, guide generation of missing image/text, and rank candidates by graph plus CLIP/BLIP similarity.

Problem

Missing modality completion methods rely on carefully designed fusion and extensive pre-training on complete data, which limits out-of-domain generalization and requires substantial resources.

Approach

Use a pre-trained LMM to automatically mine entities and relations from available modalities via chain-of-thought prompting, build a modality-specific knowledge graph, use it to guide multi-view candidate generation with diffusion models, and rank candidates by combining graph adjacency cosine similarity with CLIP and BLIP embedding similarity, all without training.

Key results

• On MM-IMDb at missing rate 0.7, Qwen-VL-7B variant reaches F1 46.3 vs 36.8 for the best baseline (MPMM) (reported 46.3 F1 on IU X-ray; MM-IMDb 0.7 F1 55.2 vs 48.5/49.1 baselines)
• On IU X-ray at missing rate 0.7, the method achieves F1 46.3 and AP 70.5 vs 36.8 F1 for the best baseline
• Removing knowledge modeling drops MM-IMDb-style OOD metrics sharply (IU X-ray F1 -17.5 at missing rate 0.7 in ablation)

Measured effects reported in the paper

Surprising findings

• Scaling the LMM from 2B to 72B and GPT-4o improves knowledge extraction, with GPT-4o giving the largest gains
• The method improves classification F1/AP at high missing rates while decreasing mean similarity score of completed modalities

Limitations

• Only image and text modalities are studied
• Hallucinations inherent to LMM may degrade generation accuracy
• Similarity score decreases even as classification performance improves

Experimental setup

Datasets: COCO-2014 MM-IMDb IU X-ray

Benchmarks: COCO-2014 MM-IMDb IU X-ray

Metrics: F1 AP mean similarity score (SS)

Baselines: MMIN DiCMoR MPMM MPLMM Baseline (remove missing)

Builds on: Chain-of-Thought CLIP BLIP Stable Diffusion XL Cheff

Closest papers by meaning

• Do More Modalities Always Help? A Geometric Perspective on Missing-Modality Robustness (NeurIPS 2026 · similarity 0.90)
• Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts (NeurIPS 2024 · similarity 0.90)
• Mitigating Pervasive Modality Absence Through Multimodal Generalization and Refinement (AAAI 2025 · similarity 0.88)
• Gradient-Guided Modality Decoupling for Missing-Modality Robustness (AAAI 2024 · similarity 0.88)
• Distilled Prompt Learning for Incomplete Multimodal Survival Prediction (CVPR 2025 · similarity 0.88)
• Probabilistic Conformal Distillation for Enhancing Missing Modality Robustness (NeurIPS 2024 · similarity 0.86)