Research map / Label-efficient and robust learning / Recognition and long-tail
Concept Distillation: Leveraging Human-Centered Explanations for Model Improvement
Avani Gupta, Saurabh Saini, P J Narayanan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DINO teacher for CAV learning (concept distillation) (ablation) | Concept loss with prototypes · ColorMNIST | accuracy (%): 26.97 -> 41.83 | Improved |
| add prototype-based intermediate-layer sensitivity (ablation) | Concept distillation · ColorMNIST | accuracy (%): 30.94 -> 50.93 | Improved |
| add local explanation (RRR) loss to concept loss | Ours · ColorMNIST | accuracy (%): 41.83 -> 50.93 | Improved |
| add local explanation (RRR) loss to concept loss | Ours · DecoyMNIST | accuracy (%): 98.58 -> 98.98 | No clear effect |
| use concept loss vs CDEP baseline | Ours+L · TextureMNIST | accuracy (%): 10.18 -> 56.57 | Improved |
| use concept sensitivity with intermediate-layer prototypes vs final-layer logit (ablation) | Concept loss · ColorMNIST | accuracy (%): 25.55 -> 41.83 | Improved |
| use varying prototypes instead of fixed prototypes (ablation) | Concept loss · ColorMNIST | accuracy (%): 40.02 -> 41.83 | Improved |
| increase teacher training data bias from 5% to 100% (ablation) | Concept distillation · ColorMNIST | student accuracy (%): 40.47 -> 23.63 | Hurt |
| 100% biased teacher vs no distillation (ablation) | Concept distillation · ColorMNIST | student accuracy (%): 26.97 -> 23.63 | Hurt |
| use DINO ViT-B/8 teacher instead of ViT-S/8 (ablation) | Concept distillation · ColorMNIST | accuracy (%): 39 -> 50.93 | Improved |
| train concept loss vs base model on bias concept TCAV | Ours · BFFHQ | TCAV score of bias concept: 0.78 -> 0.13 | Improved |
| train concept loss on ColorMNIST | Ours · ColorMNIST | TCAV score of bias concept: 0.52 -> 0.21 | Improved |
Closest papers by meaning
- Overlooked Factors in Concept-Based Explanations: Dataset Choice, Concept Learnability, and Human Capability (CVPR 2023 · similarity 0.89)
- Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors (AAAI 2021 · similarity 0.89)
- FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks (ICML 2025 · similarity 0.88)
- Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants (NeurIPS 2026 · similarity 0.88)
- FaCT: Faithful Concept Traces for Explaining Neural Network Decisions (NeurIPS 2025 · similarity 0.87)
- Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
