Research map / Probabilistic and scientific modeling / Molecules and proteins
Faithful Model Explanations through Energy-Constrained Conformal Counterfactuals
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ECCCo, a counterfactual generator that uses energy-based modelling and split conformal prediction on the black-box model itself (no surrogate) to produce counterfactuals faithful to what the model learned, plus a faithfulness metric.
Problem
Counterfactual explanations typically achieve plausibility via surrogate generative models, which decouples explanations from the black-box model and so may not faithfully describe its behaviour; fidelity alone cannot distinguish explanations.
Approach
Define faithfulness as consistency with the model's own learned conditional distribution, estimated via SGLD samples from a joint energy model; optimize counterfactuals with an energy penalty on the counterfactual and a conformal prediction set-size penalty, requiring no surrogate model.
Key results
- On California Housing with the JEM Ensemble, ECCCo+ achieves implausibility 1.31 vs 1.02-ish baseline range and unfaithfulness 1.29 vs Wachter 2.14 (JEM), showing lower unfaithfulness than baselines across tabular datasets.
- ECCCo reduces unfaithfulness substantially versus Wachter on tabular datasets, e.g., GMSC MLP Ensemble 1.20 vs 4.10 for Wachter (MLP Ensemble).
- On MNIST, ECCCo+ and REVISE achieve comparable implausibility (0.306 vs 0.301 for LeNet-5-style comparisons), with REVISE generally best on plausibility.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the energy constraint to ECCCo (rather than remove the EBM term) (ablation) | ECCCo · California Housing | Unfaithfulness: 1.70 ± 0.09 -> 1.40 ± 0.08 | Helped |
| add the energy constraint to ECCCo (rather than remove the EBM term) (ablation) | ECCCo · GMSC | Unfaithfulness: 1.31 ± 0.07 -> 1.20 ± 0.06 | Helped |
| add the energy constraint to ECCCo (rather than remove the EBM term) (ablation) | ECCCo · California Housing | Implausibility: 0.99 ± 0.08 -> 0.69 ± 0.05 | Mixed |
| add conformal prediction set-size penalty (ablation) | ECCCo · California Housing | Unfaithfulness: 1.39 ± 0.08 -> 1.40 ± 0.08 | No clear effect |
| use PCA latent-space search (ablation) | ECCCo · California Housing | Implausibility: 0.69 ± 0.05 -> 0.60 ± 0.04 | Helped |
| use PCA latent-space search (ablation) | ECCCo · MNIST | Implausibility: 0.420 ± 0.001 -> 0.306 ± 0.001 | Helped |
| use PCA latent-space search (ablation) | ECCCo · MNIST | Unfaithfulness: 0.243 ± 0.000** -> 0.246 ± 0.000* | Hurt |
| combine energy and conformal uncertainty constraints (ablation) | ECCCo · California Housing | Unfaithfulness: 4.03 ± 0.07 -> 3.69 ± 0.08** | Helped |
| use ECCCo instead of Wachter | MLP Ensemble counterfactual generation · California Housing | Unfaithfulness: 1.71 ± 0.09 -> 1.40 ± 0.08 | Helped |
| use ECCCo instead of Wachter | MLP Ensemble counterfactual generation · GMSC | Unfaithfulness: 1.31 ± 0.08 -> 1.20 ± 0.06 | Helped |
| use ECCCo+ instead of ECCCo (ablation) | MLP Ensemble counterfactual generation · GMSC | Unfaithfulness: 1.20 ± 0.06 -> 1.01 ± 0.07 | No clear effect |
Surprising findings
- REVISE, a surrogate-based plausibility method, sometimes scores better on faithfulness when the underlying model itself has learned plausible representations (e.g., JEM ensemble, LeNet-5).
Limitations
- Faithfulness metric requires gradient access and SGLD sampling, so it cannot evaluate non-differentiable models such as decision trees.
- Sensitive to energy-based modelling issues (scale sensitivity, training instability, hyperparameter sensitivity), with energy overshoot in grid search.
- Only split conformal prediction and fixed error rate were explored.
Experimental setup
Datasets: California Housing Give Me Some Credit German Credit MNIST Fashion MNIST Linearly Separable Moons Circles
Benchmarks: California Housing Give Me Some Credit MNIST Fashion MNIST German Credit Linearly Separable Moons Circles
Metrics: unfaithfulness implausibility uncertainty (set size penalty) cost redundancy validity
Baselines: Wachter REVISE Schut
Builds on: Wachter counterfactuals REVISE Schut et al. JSMA counterfactuals joint energy models split conformal prediction
Closest papers by meaning
- Model Reconstruction Using Counterfactual Explanations: A Perspective From Polytope Theory (NeurIPS 2024 · similarity 0.90)
- A New Approach to Backtracking Counterfactual Explanations: A Unified Causal Framework for Efficient Model Interpretability (ICML 2025 · similarity 0.87)
- DiCoFlex: Model-Agnostic Diverse Counterfactuals with Flexible Control (NeurIPS 2025 · similarity 0.87)
- LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching (NeurIPS 2025 · similarity 0.87)
- CF-OPT: Counterfactual Explanations for Structured Prediction (ICML 2024 · similarity 0.87)
- Promoting Counterfactual Robustness through Diversity (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.