Research map / Detection and segmentation / Open-set and few-shot detection

Interpretable Image Classification via Non-parametric Part Prototype Learning

CVPR 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Object-level representation learning · 11 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a part-based interpretable image classifier that learns class-wise non-parametric part prototypes by Sinkhorn-Knopp clustering of frozen DINO ViT features, then fine-tunes the backbone with block expansion and a patch-prototype contrastive loss, and introduces Distinctiveness and Comprehensiveness scores for ProtoPNet explanations.

Problem

Existing ProtoPNets learn parametric prototypes that often collapse onto the same repetitive object parts, producing redundant and non-comprehensive explanations.

Approach

Replace learnable prototypes with non-parametric prototypes obtained by equipartitioned, uniquely-assigned optimal-transport clustering of foreground patch features from foundation ViTs, updated as cluster means, then fine-tune the backbone (via block expansion) to ground patches to their assigned prototypes.

Key results

• On CUB-200-2011 with DINOv2 ViT-B, the method's Distinctiveness at K=5 is 94.45% versus 72.12% for EvalProtoPNet, and classification accuracy is 90.82% vs 90.06%.
• Ablation shows removing the unique-assignment and equipartition constraints drops accuracy to 9.25% and removing foreground extraction drops it to 64.29%, while the full model reaches 91.02%.
• In concept-learning experiments on CUB, the K=5 variant reaches 47.04% localization trustworthiness on ViT-B, above the 45.95% of Huang et al., with 88.18% accuracy.

Measured effects reported in the paper

Surprising findings

• Naively fine-tuning the added ViT blocks without the patch-prototype loss reduces consistency, stability and distinctiveness, so the grounding loss is needed to preserve interpretability.

Limitations

• Classification accuracy is comparable rather than clearly superior on some backbones (e.g., ViT-S and Stanford Cars/Dogs results only in supplementary).
• Concept trustworthiness is not uniformly better than the best baseline at K=3 (41.18% vs 45.95%).

Experimental setup

Datasets: CUB-200-2011 Stanford Cars Stanford Dogs LVD-142M ImageNet

Benchmarks: CUB-200-2011 Stanford Cars Stanford Dogs

Metrics: top-1 accuracy Consistency Stability Distinctiveness Score Comprehensiveness Score concept trustworthiness (localization accuracy)

Baselines: ProtoPNet Deformable ProtoPNet ProtoPool TesNet EvalProtoPNet Vanilla CBM CBM-h CEM

Builds on: ProtoPNet TesNet ProtoPool ProtoSeg Block Expansion

Closest papers by meaning

• Interpretable Image Classification with Adaptive Prototype-based Vision Transformers (NeurIPS 2024 · similarity 0.91)
• LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Vision (ICLR 2025 · similarity 0.91)
• Improving Prototypical Visual Explanations with Reward Reweighing, Reselection, and Retraining (ICML 2024 · similarity 0.90)
• ProtoArgNet: Interpretable Image Classification with Super-Prototypes and Argumentation (AAAI 2025 · similarity 0.89)
• Post-hoc Part-Prototype Networks (ICML 2024 · similarity 0.89)
• A Robust Prototype-Based Network with Interpretable RBF Classifier Foundations (AAAI 2025 · similarity 0.88)