Research map / Probabilistic and scientific modeling / Causal discovery
Multi-View Causal Representation Learning with Partial Observability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a unified contrastive identifiability framework for multi-view representation learning under partial observability, where each view nonlinearly mixes a subset of causally related latents, recovering shared content blocks up to smooth bijection with one encoder per view.
Problem
Identifying shared (content) latent variables from multiple simultaneously observed views when each view depends only on a subset of possibly causally dependent latents and mixing is nonlinear.
Approach
Uses content alignment (contrastive positive pairs) plus entropy regularization on view-specific encoders, with learnable content selectors and an L0 information-sharing regularizer, so one encoder per view recovers all shared content blocks; adds "identifiability algebra" (intersection, complement, union) via graphical criteria.
Key results
- On self-supervised disentanglement, DCI scores were 0.42 on MPI3D complex and 0.30 on 3DIdent, versus 0.11 and 0.09 for Ada-GVAE.
- Numerical experiments show shared content variables recovered with R^2 near 1 while independent style variables have R^2 near 0.
- Multi-task experiment with latent causal relations reaches 0.99 accuracy with linear and nonlinear labeling functions within 1k update steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed encoder-only contrastive method instead of Ada-GVAE | Self-supervised disentanglement · MPI-3D complex | DCI disentanglement: 0.11± 0.008 -> 0.42± 0.020 | Helped |
| Use the proposed encoder-only contrastive method instead of Ada-GVAE | Self-supervised disentanglement · 3DIdent | DCI disentanglement: 0.09± 0.019 -> 0.30± 0.04 | Helped |
| Learn a representation of shared content across views | Synthetic classification with latent causal relations · Synthetic linear and nonlinear labeling tasks | accuracy: 0.99 | Helped |
| Use the proposed method to identify content variables from multiple views | Numerical theory validation · Synthetic data with independent latents, content variables | R²: ≈ 1 | Helped |
| Use the proposed method to identify content while discarding independent styles | Numerical theory validation · Synthetic data with independent latents, style variables | R²: ≈ 0 | Hurt |
Surprising findings
- Contrastive encoder-only training outperformed the autoencoder-based Ada-GVAE on visually complex data, which the authors attribute to avoiding image reconstruction.
Limitations
- Gap between theory and practice: discrete variables and finite-sample errors are addressed only empirically.
- Minimizing the L0 information-sharing regularizer is highly non-convex; Gumbel-Softmax used as approximation.
- Requires known view-specific latent dimensions and, for some experiments, ground-truth content indices.
Experimental setup
Datasets: MPI-3D complex 3DIdent Multimodal3DIdent synthetic Gaussian latents with invertible MLP mixing
Benchmarks: MPI-3D complex 3DIdent Multimodal3DIdent synthetic numerical simulations
Metrics: R^2 (kernel ridge / logistic regression probing) DCI disentanglement classification accuracy
Baselines: Ada-GVAE
Builds on: nonlinear ICA content-style identification Ada-GVAE InfoNCE Barlow Twins
Closest papers by meaning
- A Sparsity Principle for Partially Observable Causal Representation Learning (ICML 2024 · similarity 0.93)
- Learning Linear Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity (NeurIPS 2024 · similarity 0.93)
- Causal Representation Learning Made Identifiable by Grouping of Observational Variables (ICML 2024 · similarity 0.92)
- Identifiable Latent Polynomial Causal Models through the Lens of Change (ICLR 2024 · similarity 0.90)
- Identifiability Guarantees for Causal Disentanglement from Purely Observational Data (NeurIPS 2024 · similarity 0.90)
- CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Process (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.