Research map / Label-efficient and robust learning / Recognition and long-tail
Post-hoc Part-Prototype Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a post-hoc part-prototype network that decomposes a trained model's classification head into NMF-discovered, refined part-prototypes, preserving accuracy while explaining both where and what the model looks for.
Problem
Post-hoc methods like Grad-CAM show where a model looks but not what it looks for, while existing part-prototype networks need model modification, retraining, and often lose accuracy.
Approach
Decompose the trained classification head vector into a sum of k part-prototypes obtained by NMF on backbone features, scale them by convex optimization, and refine residuals with an interpretability-constrained optimization so the decomposition exactly reconstructs the head without retraining.
Key results
- On CUB-200-2011 across five backbones, the method reports the best consistency on most backbones (e.g., 62.3 on ResNet34 vs. 53.3 for TesNet) and the best stability on most (e.g., 79.5 vs. 65.4 for TesNet on ResNet34).
- On ImageNet with ResNet34, the method keeps accuracy at 75.1% (pretrained level), while ProtoPNet gets 65.5%, ProtoDeform 63.4%, and TesNet and ProtoPool run out of memory.
- Refinement recovers the head exactly: refined prototypes match the original ResNet34 accuracy (75.1) while initial NMF prototypes reach only 16.4 to 28.4 accuracy depending on k.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use post-hoc part-prototype decomposition | ResNet34 · CUB-200-2011 | consistency: +9.4 % | Helped |
| use post-hoc part-prototype decomposition | ResNet152 · CUB-200-2011 | consistency: +9.3% | Helped |
| use post-hoc part-prototype decomposition | VGG19 · CUB-200-2011 | consistency: +5.1% | Helped |
| use post-hoc part-prototype decomposition | DenseNet121 · CUB-200-2011 | consistency: +3.8% | Helped |
| use post-hoc part-prototype decomposition | ResNet34 · CUB-200-2011 | stability: +12% | Helped |
| use post-hoc part-prototype decomposition | ResNet152 · CUB-200-2011 | stability: +11% | Helped |
| use post-hoc part-prototype decomposition | DenseNet121 · CUB-200-2011 | stability: +23.7% | Helped |
| use post-hoc part-prototype decomposition | DenseNet161 · CUB-200-2011 | stability: +19.2% | Helped |
| add prototype refinement to initial prototypes (ablation) | ResNet34, ViT-Base, and CoC-Tiny · ImageNet | accuracy: 16.4 -> 75.1; 50.8 -> 76.3; 45.1 -> 71.9 | Helped |
| increase prototype count from k=3 to k=10 before refinement (ablation) | ImageNet models · ImageNet | accuracy: Res: 16.4 -> 28.4; ViT: 50.8 -> 54.8; CoC: 45.1 -> 54.3 | No clear effect |
| use dynamic optimization with interpretability constraints instead of naive residual distribution (ablation) | prototype refinement · CUB-200-2011 | consistency and stability | Helped |
| add residual refinement with interpretability constraints to initial prototypes (ablation) | prototype refinement with k=5 and k=10 · CUB-200-2011 | consistency and stability | Helped |
Surprising findings
- Refinement with interpretability constraints improved consistency and stability over the initial NMF prototypes at k=5 and k=10, not only preserving them.
Limitations
- Explanations are limited by the quality of the trained black-box model's features; if features are not distinct, prototypes are hard to discover.
- Accuracy on CUB is not tuned for SOTA fine-tuning; the authors note fine-tuning tricks could further increase accuracy.
- Negative feature values from GeLU/ViT must be clipped to zero to apply NMF, which is a simplification.
Experimental setup
Datasets: CUB-200-2011 ImageNet Stanford Cars Stanford Dogs
Benchmarks: CUB-200-2011 ImageNet Stanford Cars Stanford Dogs
Metrics: consistency score stability score accuracy top-1 accuracy
Baselines: ProtoPNet ProtoTree Deformable ProtoPNet TesNet ProtoPool ProtoPNet (ProtoEval)
Builds on: ProtoPNet Grad-CAM Non-negative Matrix Factorization
Closest papers by meaning
- Interpretable Image Classification via Non-parametric Part Prototype Learning (CVPR 2025 · similarity 0.89)
- LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Vision (ICLR 2025 · similarity 0.86)
- Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Time (CVPR 2025 · similarity 0.85)
- Keep the Faith: Faithful Explanations in Convolutional Neural Networks for Case-Based Reasoning (AAAI 2024 · similarity 0.85)
- Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanations (ICML 2025 · similarity 0.85)
- How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.