Research map / Probabilistic and scientific modeling / Causal discovery
Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes aligning causal representations learned by CRL methods to interpretable concepts using few concept labels and no interventions, via group-lasso (linear, finite-sample) and kernelized (asymptotic) permutation estimators, evaluated on toy, Action/Temporal Sparsity and Temporal Causal3DIdent data.
Problem
Concept bottleneck models learn spurious or impure concepts when concepts are correlated, and require many expensive concept labels; existing fixes assume independent concepts or need interventions.
Approach
Use pretrained causal representation learning encoders to obtain disentangled latent causal variables from unlabeled data, then learn a permutation plus element-wise alignment to concepts with a group-lasso or kernelized group-lasso estimator, requiring few concept labels and no interventions, with theoretical guarantees.
Key results
- On Action Sparsity with 20 labels, the DMS-VAE linear estimator reaches label accuracy 0.77 and OIS 0.63, versus 0.60 and 0.92 for CBM.
- On Temporal Causal3DIdent with 10000 points, CITRISVAE spline reaches label accuracy 0.83 and OIS 0.09, versus 0.78 and 0.18 for CBM.
- The estimators reach perfect NIS scores and are reported to run significantly faster than the neural-network and CBM baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use linear alignment instead of CEM | Action Sparsity downstream classifier, n=20 · Action Sparsity Dataset (n=20) | Label accuracy: 0.66 { $\pm$ 0.02} -> 0.77 { $\pm$ 0.03} | Helped |
| Use linear alignment instead of CEM | Action Sparsity downstream classifier, n=20 · Action Sparsity Dataset (n=20) | OIS: 0.90 { $\pm$ 0.03} -> 0.63 { $\pm$ 0.01} | Helped |
| Use spline alignment instead of HardCBM | Action Sparsity downstream classifier, n=10000 · Action Sparsity Dataset (n=10000) | Label accuracy: 0.89 { $\pm$ 0.00} -> 0.86 { $\pm$ 0.01} | Hurt |
| Use spline alignment instead of HardCBM | Action Sparsity downstream classifier, n=10000 · Action Sparsity Dataset (n=10000) | OIS: 0.06 { $\pm$ 0.00} -> 0.12 { $\pm$ 0.00} | Hurt |
| Use linear alignment instead of CEM | Temporal Causal3DIdent downstream classifier, n=20 · Temporal Causal3DIdent Dataset (n=20) | Label accuracy: 0.64 { $\pm$ 0.04} -> 0.74 { $\pm$ 0.06} | Helped |
| Use RFF alignment instead of CEM | Temporal Causal3DIdent downstream classifier, n=20 · Temporal Causal3DIdent Dataset (n=20) | OIS: 0.97 { $\pm$ 0.05} -> 0.65 { $\pm$ 0.01} | Helped |
| Use spline alignment instead of CEM | Temporal Causal3DIdent downstream classifier, n=10000 · Temporal Causal3DIdent Dataset (n=10000) | Label accuracy: 0.78 { $\pm$ 0.05} -> 0.83 { $\pm$ 0.03} | Helped |
| Use spline alignment instead of CEM | Temporal Causal3DIdent downstream classifier, n=10000 · Temporal Causal3DIdent Dataset (n=10000) | OIS: 0.49 { $\pm$ 0.01} -> 0.09 { $\pm$ 0.00} | Helped |
| Replace linear features with spline features (ablation) | DMS-VAE alignment · Action Sparsity Dataset | Label accuracy: 0.77 { $\pm$ 0.03} -> 0.72 { $\pm$ 0.03} (n=20); 0.84 { $\pm$ 0.01} -> 0.86 { $\pm$ 0.01} (n=10000) | Mixed |
| Replace linear features with spline features (ablation) | CITRISVAE alignment · Temporal Causal3DIdent Dataset | Label accuracy: 0.74 { $\pm$ 0.06} -> 0.74 { $\pm$ 0.06} (n=20); 0.81 { $\pm$ 0.04} -> 0.83 { $\pm$ 0.03} (n=10000) | Mixed |
Surprising findings
- The estimator recovers the correct permutation even under strong correlations between concepts, beyond the theoretical assumptions.
- Baselines such as CBM, CEM and HardCBM perform well on mean concept accuracy in the Action/Temporal sparsity datasets, though the proposed estimator still wins with few labels.
Limitations
- Theory assumes continuous causal variables and concepts; binary concepts are only evaluated empirically.
- Assumes human concepts coincide with the causal variables recovered by CRL, and that CRL achieves perfect disentanglement.
- Theoretical analysis requires low correlation between learned representations, stronger than what the experiments need.
Experimental setup
Datasets: Toy dataset (synthetic) Action Sparsity dataset Temporal Sparsity dataset Temporal Causal3DIdent
Benchmarks: Toy dataset Action Sparsity dataset Temporal Causal3DIdent
Metrics: permutation error (MPE) label accuracy Oracle Impurity Score (OIS) Non-linear Impurity Score (NIS) computation time
Baselines: CBM CEM HardCBM Pearson correlation Spearman correlation linear sum assignment
Builds on: concept bottleneck models causal representation learning group lasso iVAE CITRIS
Closest papers by meaning
- Causally Reliable Concept Bottleneck Models (NeurIPS 2025 · similarity 0.91)
- Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations (ICLR 2024 · similarity 0.91)
- A theoretical design of concept sets: improving the predictability of concept bottleneck models (NeurIPS 2024 · similarity 0.91)
- Stochastic Concept Bottleneck Models (NeurIPS 2024 · similarity 0.90)
- An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations (NeurIPS 2025 · similarity 0.90)
- Counterfactual Concept Bottleneck Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.