Research map / Probabilistic and scientific modeling / Causal discovery
From Black-box to Causal-box: Towards Building More Interpretable Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces causal interpretability for model classes, shows blackbox and concept-based predictors are not causally interpretable in general, and gives a graphical criterion and unique maximal feature set for building counterfactually interpretable models.
Problem
Counterfactual queries about a deep model's predictions (what if a high-level feature were changed) can be inconsistent across models that agree on observational data, so such queries may be unanswerable from the model class; blackbox and concept-based models are not known to be causally interpretab
Approach
Define causal interpretability of a model class as uniqueness of counterfactual query values across all ASCMs in the class that match the observational distribution; prove blackbox classes never satisfy it, give a complete graphical criterion (T subset of W union ND(W)) for feature-subset (GCP) models, and show a unique maximal feature set for a set of queries, yielding an accuracy-interpretabilit
Key results
- Blackbox model classes are never causally interpretable for any latent causal graph and any query Q(W) (Proposition on non-interpretability of BP).
- On BarMNIST, the model using T={B,D,C} is most accurate but has high counterfactual error for changing digit, while T={D,C} is the unique maximal causally interpretable set for that query.
- On CelebA, a model using non-descendants of smiling (smile, gender) predicts higher attractiveness under the counterfactual 'had they smiled', whereas baselines using descendants (cheekbones, opened mouth) often do not.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use features {B, D, C} instead of {D, C} (ablation) | GCP model · BarMNIST | accuracy | Helped |
| Use features {B, D} instead of {D, C} (ablation) | GCP model · BarMNIST | accuracy | Mixed |
| Use features {D} instead of {D, C} (ablation) | GCP model · BarMNIST | accuracy | Hurt |
| Include descendant feature B when predicting (ablation) | GCP model · BarMNIST, changing digit | counterfactual estimation error | Hurt |
| Use features {D, C} or {D} rather than including descendant feature B (ablation) | GCP model · BarMNIST, changing digit | counterfactual estimation | Helped |
| Use only feature B rather than features B and C (ablation) | GCP model · BarMNIST, changing bar | counterfactual estimation | Helped |
| Use non-descendant features smiling and gender rather than adding cheekbones and opened mouth (ablation) | GCP model · CelebA, changing smiling | counterfactual attractiveness prediction | Helped |
| Add descendant feature cheekbones to smiling and gender (ablation) | CelebA baseline 1 · CelebA, changing smiling | counterfactual interpretability | Hurt |
| Add descendant features cheekbones and opened mouth to smiling and gender (ablation) | CelebA baseline 2 · CelebA, changing smiling | counterfactual interpretability | Hurt |
Surprising findings
- Concept-based models that use all observed features are not causally interpretable in general, contrary to prior belief.
- Adding features that are descendants of the query variable destroys causal interpretability, even if they improve accuracy.
Limitations
- Identifiability results assume a known or partially known latent causal graph structure and features, with the admissibility criterion depending on the descendant structure.
- Experiments are small-scale (synthetic BarMNIST and CelebA with binary attributes) and use a single label.
Experimental setup
Datasets: BarMNIST (synthetic, 60,000 images) CelebA
Benchmarks: BarMNIST CelebA
Metrics: counterfactual estimation error vs ground truth prediction accuracy average difference between counterfactual and factual predic
Baselines: blackbox model concept-based model with all features (CP) GCP with descendant features (baseline 1 and 2 on CelebA)
Builds on: structural causal model concept bottleneck models do-calculus augmented SCM
Closest papers by meaning
- Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning (ICLR 2025 · similarity 0.90)
- A New Approach to Backtracking Counterfactual Explanations: A Unified Causal Framework for Efficient Model Interpretability (ICML 2025 · similarity 0.88)
- Counterfactual Concept Bottleneck Models (ICLR 2025 · similarity 0.87)
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability? (NeurIPS 2025 · similarity 0.86)
- LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching (NeurIPS 2025 · similarity 0.86)
- Towards Characterizing Domain Counterfactuals for Invertible Latent Causal Models (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.