Research map / Probabilistic and scientific modeling / Causal discovery
Identifying Representations for Intervention Extrapolation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Rep4Ex, an identifiable representation learning method (autoencoder with a linear-invariance MMR regularizer) plus control-function regression that enables nonlinear extrapolation of intervention effects on exogenous actions outside the training support.
Problem
Predicting the effect of interventions on exogenous action variables that lie outside the training support, when observed features are a nonlinear mixing of latent predictors and the action-to-latent effect is linear.
Approach
Identify the unmixing function up to an affine transformation using a linear invariance constraint (residuals of regressing the encoder output on A have zero conditional mean given A), enforced via a maximum moment restriction regularizer on an autoencoder; then apply control-function additive regression to extrapolate E[Y | do(A=a*)].
Key results
- AE-MMR achieves R-squared approaching 1 as intervention strength alpha increases, while AE-Vanilla and VAE yield significantly lower R-squared.
- In one-dimensional experiments, Rep4Ex-CF extrapolates beyond the training support where the MLP baseline fails, achieving nearly perfect extrapolation at gamma=1.2.
- In multi-dimensional experiments (d in {2,4,10}), Rep4Ex-CF achieves markedly lower MSE than MLP and performs on par with the oracle that uses true latents.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Enforce linear invariance with MMR regularization in the autoencoder | Autoencoder representation learning · Synthetic unmixing experiments | R-squared | Helped |
| Use a vanilla autoencoder without MMR regularization | Autoencoder representation learning · Synthetic unmixing experiments | R-squared | Hurt |
| Use a variational autoencoder instead of the MMR-regularized autoencoder | Autoencoder representation learning · Synthetic unmixing experiments | R-squared | Hurt |
| Use Rep4Ex-CF instead of MLP regression on A | Intervention extrapolation · One-dimensional A | extrapolation performance | Helped |
| Increase the training support parameter γ | Rep4Ex-CF · One-dimensional A | extrapolation performance | Helped |
| Use Rep4Ex-CF instead of MLP regression on A | Intervention extrapolation · Multi-dimensional A | mean squared error | Helped |
| Learn the representation rather than use the true latent predictors | Rep4Ex-CF · Multi-dimensional A | mean squared error | No clear effect |
Surprising findings
- Regressing Y on A non-parametrically cannot identify the interventional effect outside the support of A, even with exogenous A (Proposition 1).
Limitations
- Requires linearity of the effect of A on Z and full support of the noise V, which may not hold in practice.
- Identifiability is only up to an affine transformation and assumes noise-free observed features X = g(Z).
- Validation is limited to synthetic experiments.
Experimental setup
Datasets: synthetic data from structural causal models
Benchmarks: synthetic SCM experiments
Metrics: R-squared mean squared error
Baselines: AE-Vanilla VAE MLP Rep4Ex-CF-Oracle AE-MMR-Oracle
Builds on: maximum moment restriction control function approach nonlinear ICA instrumental variables
Closest papers by meaning
- Learning Linear Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity (NeurIPS 2024 · similarity 0.89)
- Linear Causal Representation Learning from Unknown Multi-node Interventions (NeurIPS 2024 · similarity 0.89)
- A Sparsity Principle for Partially Observable Causal Representation Learning (ICML 2024 · similarity 0.88)
- Causal Representation Learning Made Identifiable by Grouping of Observational Variables (ICML 2024 · similarity 0.87)
- Identifiable Latent Polynomial Causal Models through the Lens of Change (ICLR 2024 · similarity 0.87)
- Learning Joint Interventional Effects from Single-Variable Interventions in Additive Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.