Research map / Probabilistic and scientific modeling / Causal discovery

Optimization-based Causal Estimation from Heterogeneous Environments

NeurIPS 2024 · Probabilistic and scientific modeling › Causal discovery · approach: Latent-variable causal estimation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CoCo, a multi-environment optimization objective (norm of gradient-times-parameter of the risk) whose unique solution under heterogeneity and invariance is the causal coefficient vector, with linear and nonlinear extensions.

Problem

Estimating which covariates are direct causes of an outcome and their causal effect when observational covariates contain spurious associations (confounding, colliders, reverse causality) that bias ERM/MLE and IRM estimates.

Approach

Uses directional derivatives of the risk along the causal coefficient direction to derive a tractable objective, the norm of (gradient of risk Hadamard parameter) averaged over heterogeneous environments; the intersection of per-environment optima shrinks to the causal coefficient, with a risk regularizer and known-exogenous-variable modification to remove the zero solution. Shown to be a strictly

Key results

• On the two-environment linear case study, CoCo converges to (3.001, 2.004, 0.000) versus the true (3,2,0), while IRMv1 with random initialization gives (-0.842, -0.390, 0.621) at lambda=1.
• On ColoredMNIST with noised labels, CoCo reaches 74.7% test accuracy (0.3 std), above Fishr (73.8%) and MM-REx (66.1%) as reported in the original papers.
• Simulations show CoCo has lower MAE for causal coefficients than RVP, V-REx, IRM and ERM across two and four environments.

Measured effects reported in the paper

Surprising findings

• The IRMv1 optimum from the true causal coefficient initialization is not the causal coefficient, showing IRM's minimizer can be a biased causal estimate even when the causal solution is an optimum.
• Under the stated invariance and heterogeneity conditions, ERM solutions can coincide with spurious non-causal points that CoCo must exclude.

Limitations

• Requires invariance of the conditional expectation of the outcome given its causal parents and sufficiently heterogeneous environments (effectiveness assumption).
• Requires convex risk in the derivations; the zero vector and ERM solution remain optima without extra modifications such as known exogenous variables or risk regularization, which introduces potential estimation bias.
• Assumes no unmeasured confounding between true causes and outcome; the paper's table marks this as not handled.

Experimental setup

Datasets: ColoredMNIST iWildCam synthetic linear SEM data

Benchmarks: ColoredMNIST iWildCam synthetic linear SEM

Metrics: test accuracy mean absolute error of causal coefficients

Baselines: ERM IRM V-REx RVP Fishr MM-REx ICP

Builds on: invariant risk minimization invariant causal prediction directional derivative first-order optimality

Closest papers by meaning

• Information-Theoretic Causal Discovery and Intervention Detection over Multiple Environments (AAAI 2023 · similarity 0.84)
• Learning Causal Structures Using Regression Invariance (NeurIPS 2017 · similarity 0.84)
• Conditions and Assumptions for Constraint-based Causal Structure Learning (ICML 2023 · similarity 0.83)
• Causal-learn: Causal Discovery in Python (NeurIPS 2024 · similarity 0.82)
• Causal Inference with Conditional Front-Door Adjustment and Identifiable Variational Autoencoder (ICLR 2024 · similarity 0.81)
• Federated Causality Learning with Explainable Adaptive Optimization (AAAI 2024 · similarity 0.81)