Research map / Probabilistic and scientific modeling / Causal discovery
A Sparsity Principle for Partially Observable Causal Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves identifiability of causal latents under unpaired, instance-dependent partial observability via a sparsity constraint on the encoder output, for linear mixing and for piecewise linear mixing with Gaussian latents, and implements both as constrained optimization.
Problem
Causal representation learning when each observation measures only an instance-dependent, unknown subset of the latent causal variables, with unpaired observations and no fixed view/domain structure.
Approach
Use an L0 sparsity principle on the learned representation: with zero reconstruction error and sufficient support-index variability, sparsity matches the ground-truth masked variables and yields element-wise identifiability. For piecewise linear mixing, Gaussianity of latents within observed-pattern groups (enforced via skewness/kurtosis regularizers) makes the composition affine.
Key results
- Linear mixing with Gaussian SCM, n=10, k=1, rho=50%: MCC 0.996 +- 0.001, degrading to 0.714 at n=40.
- Multiple Balls MCC stays above 0.90 for 2, 5, 8 balls (e.g., 0.963 missing-ball and 0.946 masked-position with 2 balls).
- PartialCausal3DIdent average MCC 0.832 across 7 object classes, with all classes above 0.80.
- Piecewise linear method performance drops with more variables or complex mixing; a mask-informed oracle performs better, showing the Gaussianity regularizer is the bottleneck.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the number of causal variables from 5 to 40 | Sparsity method with linear mixing · Numerical linear Gaussian SCM, k=1, 50% measured | MCC: 0.997$\pm$0.002 -> 0.714$\pm$0.153 | Hurt |
| Increase causal graph degree from k=0 to k=3 | Sparsity method with linear mixing · Numerical linear Gaussian SCM, n=10, 50% measured | MCC: 0.998$\pm$0.001 -> 0.793$\pm$0.142 | Hurt |
| Increase measured-variable ratio from 1 var to 75% | Sparsity method with linear mixing · Numerical linear Gaussian SCM, n=10, k=1 | MCC: 0.998 $\pm$0.002 -> 0.877 $\pm$0.096 | Hurt |
| Increase number of balls from 2 to 8 | Sparsity method · Multiple Balls, missing ball | MCC: 0.963$\pm$0.012 -> 0.928$\pm$0.004 | Hurt |
| Use missing-ball rather than masked-position data | Sparsity method · Multiple Balls, b=2 | MCC: 0.946$\pm$0.005 -> 0.963$\pm$0.012 | Helped |
| Increase masking distance δ | Sparsity method with piecewise linear mixing · Numerical piecewise-linear experiments | MCC | Helped |
| Increase mixing-function complexity (more layers) | Sparsity method with piecewise linear mixing · Numerical piecewise-linear experiments | MCC | Hurt |
| Increase causal graph degree k | Sparsity method with piecewise linear mixing · Numerical piecewise-linear experiments | MCC | No clear effect |
| Use oracle mask information to set low variance for masked variables (ablation) | Sparsity method with piecewise linear mixing · Numerical piecewise-linear experiments | MCC | Helped |
| Overestimate the encoder output dimension beyond n (ablation) | Sparsity method with linear mixing · Numerical experiments | MCC | No clear effect |
| Increase the measured-variable ratio ρ | Sparsity method with piecewise linear mixing · Numerical piecewise-linear experiments | MCC | Hurt |
Surprising findings
- The estimated skewness/kurtosis regularizers fail to enforce Gaussianity well, so the piecewise linear method lags an oracle that knows the masks.
- A nonlinear mixing function can break sparsity-based identifiability (counterexample), so linearity or piecewise linearity is needed.
Limitations
- Requires knowing the group (observed mask pattern) of each sample for the piecewise linear theorem during training.
- Piecewise linear mixing is a restricted function class, and the Gaussianity constraint is hard to satisfy empirically.
- Sufficient support index variability requires each variable to be masked in some pattern and unmasked in others; identifiability performance degrades with more variables and denser graphs.
Experimental setup
Datasets: Causal3DIdent Multiple Balls (new synthetic) PartialCausal3DIdent (derived from Causal3DIdent)
Benchmarks: Multiple Balls PartialCausal3DIdent simulated linear and piecewise linear numerical data
Metrics: MCC
Baselines: oracle skewness/kurtosis variant
Builds on: Lachapelle et al. sparse multitask disentanglement Kivva et al. identifiability of mixtures of Gaussians with piecewise linear mixi Cooper constrained optimization Causal3DIdent
Closest papers by meaning
- Multi-View Causal Representation Learning with Partial Observability (ICLR 2024 · similarity 0.93)
- Causal Representation Learning Made Identifiable by Grouping of Observational Variables (ICML 2024 · similarity 0.93)
- Learning Linear Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity (NeurIPS 2024 · similarity 0.92)
- Identifiable Latent Polynomial Causal Models through the Lens of Change (ICLR 2024 · similarity 0.91)
- Identifiability Guarantees for Causal Disentanglement from Purely Observational Data (NeurIPS 2024 · similarity 0.90)
- Causal Representation Learning from Multiple Distributions: A General Setting (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.