Research map / Label-efficient and robust learning / Recognition and long-tail
I-CEE: Tailoring Explanations of Image Classification Models to User Expertise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes I-CEE, a framework that selects example images with GradCAM explanations tailored to a user's concept-level expertise, improving simulatability in simulated users and a 100-person user study.
Problem
Explanation-by-examples for image classifiers are typically one-size-fits-all, ignoring user expertise, which limits how well users can simulate the model's decisions.
Approach
Model each user with a concept-space expertise vector learned from their annotations, then select training examples and local explanations whose information gain (drop in user's predicted probability of the model label when the explanation is shown) is largest, with a hypercorrection-motivated selection rule.
Key results
- On GTSRB, the human user study showed an 11.5% simulatability accuracy improvement over Bayesian Teaching (p=0.007) with N=100 participants.
- On CUB-200-2011 with human users, no significant overall difference was found, but on inaccurate-prediction test images users reached 46.3% vs 40.3% for Bayesian Teaching.
- In simulation on CUB-200-2011, the method achieved the best performance after 20% of examples, while on CIFAR-100 it underperformed EGL and DWM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Tailor selected examples to each user's expertise rather than using examples selected for a different user's expertise (ablation) | I-CEE simulated users · Synthetic dataset | simulatability accuracy | Helped |
| Use examples selected for a different user's expertise (ablation) | I-CEE simulated users · Synthetic dataset | simulatability accuracy | Hurt |
| Use I-CEE instead of Bayesian Teaching | human participants · GTSRB human study | simulatability accuracy: +11.5\% (p=0.007) | Helped |
| Use I-CEE instead of Bayesian Teaching | human participants · CUB-200-2011 human study | simulatability accuracy | No clear effect |
| Use I-CEE instead of Bayesian Teaching | human participants · CUB-200-2011, target-model inaccurate predictions | accuracy: 40.3\% -> 46.3\% | Helped |
| Use I-CEE instead of Bayesian Teaching | human participants · CUB-200-2011 human study | subjective understanding rating: 5.02 -> 5.14 | No clear effect |
| Use I-CEE instead of Bayesian Teaching | human participants · GTSRB human study | subjective understanding rating | Helped |
| Increase the concept count from m=16 to m=128 (ablation) | I-CEE simulated user model · CIFAR-100 | accuracy: 85.00 $\pm$ 0.50 -> 96.5$\pm$ 0.23 | Helped |
| Increase the concept count from m=16 to m=128 (ablation) | I-CEE simulated user model · CUB-200-2011 | accuracy: 25.75 $\pm$ 0.78 -> 65.15 $\pm$ 0.60 | Helped |
| Increase the concept count from m=8 to m=64 (ablation) | I-CEE simulated user model · GTSRB | accuracy: 89.17$\pm$ 0.34 -> 100.0 $\pm$ 0.10 | Helped |
| Use I-CEE instead of EGL or DWM | simulated users · CIFAR-100 | simulatability accuracy | Hurt |
| Use I-CEE instead of the other baselines | simulated users · CUB-200-2011 and GTSRB | simulatability accuracy | Helped |
Surprising findings
- On CIFAR-100 the method was inferior to EGL and DWM in simulation, attributed to low-resolution, noisy explanations.
- On CUB-200-2011 the human study showed no significant difference in overall simulatability or subjective understanding.
Limitations
- Expertise estimation uses a concept-based model only; more complex or LLM-based user models are left for future work.
- Does not account for sample complexity of estimating user expertise from few real-user annotations.
- Evaluated on only a few datasets; results depend on task domain, and CIFAR-100 and CUB results were mixed.
Experimental setup
Datasets: CIFAR-100 CUB-200-2011 GTSRB 3D shapes-based synthetic dataset
Benchmarks: CIFAR-100 CUB-200-2011 GTSRB Synthetic dataset
Metrics: simulatability accuracy subjective understanding Likert rating feature agreement
Baselines: Bayesian Teaching EGL EGL-Shift DWM random sampling
Builds on: Bayesian Teaching TCAV concept discovery Expected Gradient Length density-weighted active learning hypercorrection effect
Closest papers by meaning
- Counterfactual Explanations for Misclassified Images: How Human and Machine Explanations Differ (Abstract Reprint) (AAAI 2024 · similarity 0.88)
- Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics (NeurIPS 2024 · similarity 0.86)
- SLICE: Stabilized LIME for Consistent Explanations for Image Classification (CVPR 2024 · similarity 0.85)
- Explaining in Diffusion: Explaining a Classifier with Diffusion Semantics (CVPR 2025 · similarity 0.85)
- Enhancing Performance of Explainable AI Models with Constrained Concept Refinement (ICML 2025 · similarity 0.85)
- Representational Difference Explanations (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.