Research map / Probabilistic and scientific modeling / Causal discovery

Towards the Causal Complete Cause of Multi-Modal Representation Learning

ICML 2025 · Probabilistic and scientific modeling › Causal discovery · approach: Causal representation identifiability · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Causal Complete Cause (C^3), a probabilistic measure of causal sufficiency and necessity for multi-modal representations, with identifiability under non-exogeneity and non-monotonicity via an attention-based instrumental variable, and a twin-network C^3 risk used as plug-and-play regularizer (C^3R).

Problem

Multi-modal learning representations learned by consistency or specificity objectives may be causally insufficient or unnecessary, and existing causal sufficiency/necessity theory relies on exogeneity and monotonicity assumptions that fail under spurious correlations and cross-modal conflicts.

Approach

Define C^3 as the probability a representation is causally sufficient and necessary, show identifiability without exogeneity/monotonicity using a self-attention instrumental variable V, and estimate C^3 risk with a twin network (real-world branch plus gradient-based hypothetical counterfactual branch) to regularize any MML model.

Key results

• QMF+C^3R reaches 77.58 average accuracy on NYU Depth V2 under 50% Gaussian noise with 0 noise-level, versus 70.09 for QMF alone.
• LCKD+C^3R improves Enhancing Tumour, Tumour Core and Whole Tumour scores on BraTS across all 15 missing-modality combinations, e.g., up to +7.16 points for Tumour Core with only T2 available.
• C^3R reports higher correlation with sufficient and necessary causes and lower correlation with spurious correlations than baselines (Figure-based analysis).

Measured effects reported in the paper

Limitations

• Theoretical guarantees rely on approximations (e.g., Monte Carlo approximation of V, local invertibility, approximate intervention via KL-bounded gradient perturbation).
• Experiments use a small three-layer MLP representation learner and a synthetic dataset, limiting evidence on large-scale models.

Experimental setup

Datasets: NYU Depth V2 SUN RGB-D UPMC FOOD-101 MVSA BraTS MMLSynData

Benchmarks: NYU Depth V2 SUN RGB-D UPMC FOOD-101 MVSA BraTS

Metrics: accuracy (average and worst-case) Dice score on BraTS segmentation correlation score with SNC/SC/NC/SP representations

Baselines: CLIP ALIGN MaPLe CoOp VPT Late fusion ConcatMML AlignMML MMTM TMC LCKD UniCODE

Builds on: causal sufficiency and necessity (probability of necessity and sufficiency) Multi-modal Concatenation / late fusion baselines instrumental variable methods

Closest papers by meaning

• Learning Linear Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity (NeurIPS 2024 · similarity 0.89)
• Causal Representation Learning Made Identifiable by Grouping of Observational Variables (ICML 2024 · similarity 0.88)
• The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability? (NeurIPS 2025 · similarity 0.88)
• Multi-View Causal Representation Learning with Partial Observability (ICLR 2024 · similarity 0.88)
• A Sparsity Principle for Partially Observable Causal Representation Learning (ICML 2024 · similarity 0.87)
• Aligning Multimodal Representations through an Information Bottleneck (ICML 2025 · similarity 0.87)