Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Modeling Uncertainties of Self-Explaining Neural Networks via Conformal Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes unSENN, a conformal prediction framework that gives distribution-free uncertainty sets for both the concept explanations and the final labels of self-explaining networks (concept bottleneck models and prototype-based networks).
Problem
Self-explaining neural networks produce predictions and concept explanations without distribution-free uncertainty quantification for either output, and no link between explanation confidence and final prediction confidence.
Approach
Design non-conformity scores on top-K concept importance scores (with or without ground-truth concepts), derive concept prediction sets via split conformal prediction, and transfer these sets to label sets by solving an adversarial-attack-style optimization over the concept-space perturbation ball, with coverage proofs under exchangeability.
Key results
- On MNIST at miscoverage 0.1, unSENN label coverage is 0.988 with average set size 1.00, versus 0.985/1.01 for Naive and 0.903/0.90 for Vanilla.
- On CIFAR-100 Super-class at miscoverage 0.1, unSENN label coverage is 0.847 with average set size 1.03, versus 0.786/1.28 for Naive and 0.805/1.15 for Vanilla.
- Concept conformal sets keep the concept error rate near the target miscoverage on MNIST and CIFAR-100 (e.g., 0.094 at 0.1 on MNIST), while the Naive baseline has much lower error on MNIST but larger sets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use unSENN instead of Naive for concept conformal sets | Concept bottleneck model · MNIST concept sets (ε=0.1) | concept error rate: $0.003 \pm 0.000 $ -> $0.094 \pm 0.002$ | Hurt |
| Use unSENN instead of Naive for concept conformal sets | Concept bottleneck model · CIFAR-100 Super-class concept sets (ε=0.05) | concept error rate: $0.073 \pm 0.001 $ -> $0.044 \pm 0.002 $ | Helped |
| Use unSENN instead of Naive for concept conformal sets | Concept bottleneck model · CIFAR-100 Super-class concept sets (ε=0.15) | concept error rate: $0.121 \pm 0.001 $ -> $0.137 \pm 0.004 $ | Hurt |
| Use unSENN instead of Naive for label prediction sets | Concept bottleneck model · MNIST label prediction sets (ε=0.1) | label coverage: $0.985 \pm 0.002$ -> $0.988 \pm 0.002 $ | Helped |
| Use unSENN instead of Naive for label prediction sets | Concept bottleneck model · MNIST label prediction sets (ε=0.1) | average set size: $1.01 \pm 0.00$ -> $1.00 \pm 0.00$ | Helped |
| Use unSENN instead of Vanilla for label prediction sets | Concept bottleneck model · MNIST label prediction sets (ε=0.1) | label coverage: $0.903 \pm 0.001$ -> $0.988 \pm 0.002 $ | Helped |
| Use unSENN instead of Naive for label prediction sets | Concept bottleneck model · CIFAR-100 Super-class label prediction sets (ε=0.1) | label coverage: $0.786 \pm 0.004$ -> $0.847 \pm 0.002 $ | Helped |
| Use unSENN instead of Naive for label prediction sets | Concept bottleneck model · CIFAR-100 Super-class label prediction sets (ε=0.1) | average set size: $1.28 \pm 0.02$ -> $1.03 \pm 0.00$ | Helped |
| Use unSENN instead of Vanilla for label prediction sets | Concept bottleneck model · CIFAR-100 Super-class label prediction sets (ε=0.1) | label coverage: $0.805 \pm 0.006$ -> $0.847 \pm 0.002 $ | Helped |
| Use unSENN instead of Naive for prototype-based label prediction sets | Prototype-based self-explaining network · MNIST prototype-based label sets (ε=0.1) | label coverage: $0.922 \pm 0.002$ -> $0.984 \pm 0.001 $ | Helped |
| Use unSENN instead of Naive for prototype-based label prediction sets | Prototype-based self-explaining network · MNIST prototype-based label sets (ε=0.1) | average set size: $1.11 \pm 0.00$ -> $1.00 \pm 0.00$ | Helped |
| Use a larger calibration set (ablation) | unSENN · MNIST and CIFAR-100 Super-class concept sets (ε=0.1) | prediction-set stability | Helped |
Surprising findings
- On MNIST the Naive concept baseline has much lower error (about 0.003) than the target miscoverage, so it is over-conservative relative to unSENN's concept sets there.
Limitations
- Coverage guarantee relies on exchangeability of calibration and test data.
- Label set construction requires iterative adversarial optimization over all labels, adding computational cost.
- Evaluated only on two datasets (CIFAR-100 Super-class and MNIST range task) with small models.
Experimental setup
Datasets: CIFAR-100 Super-class MNIST
Benchmarks: CIFAR-100 Super-class MNIST
Metrics: concept error rate label coverage average set size running time
Baselines: Naive threshold set construction Vanilla conformal predictor
Builds on: conformal prediction concept bottleneck models prototype-based self-explaining networks
Closest papers by meaning
- Neuron Explanations for Conformal Prediction (Student Abstract) (AAAI 2025 · similarity 0.90)
- Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanations (ICML 2025 · similarity 0.89)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.87)
- Explain Yourself, Briefly! Self-Explaining Neural Networks with Concise Sufficient Reasons (ICLR 2025 · similarity 0.87)
- Q-SENN: Quantized Self-Explaining Neural Networks (AAAI 2024 · similarity 0.86)
- Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspective (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.