Research map / Label-efficient and robust learning / Recognition and long-tail
PRISM: Prototype-based Reasoning with Inter-modal Semantic Mining for Interpretable Image Recognition
Anni Yu, Yu-Bin Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text-guided alignment loss Lalign (ablation) | PRISM (Lce + Lorth) · CUB-200-2011, ViT-B/16 | consistency (%): 13.5 -> 96.3 | Improved |
| add Lalign (ablation) | PRISM (Lce + Lorth) · CUB-200-2011, ViT-B/16 | stability (%): 51.2 -> 77.5 | Improved |
| add Lalign (ablation) | PRISM (Lce + Lorth) · CUB-200-2011, ViT-B/16 | top-1 accuracy (%): 87.4 -> 87.5 | No clear effect |
| add compactness loss Lcomp on top of Lalign (ablation) | PRISM · CUB-200-2011, ViT-B/16 | consistency (%): 96.3 -> 75.8 | Hurt |
| add compactness loss Lcomp on top of Lalign (ablation) | PRISM · CUB-200-2011, ViT-B/16 | top-1 accuracy (%): 87.5 -> 87.8 | Improved |
| add Lcomp alone without Lalign (ablation) | PRISM (Lce + Lorth) · CUB-200-2011, ViT-B/16 | top-1 accuracy (%): 87.4 -> 86.8 | Hurt |
| add Lcomp alone without Lalign (ablation) | PRISM (Lce + Lorth) · CUB-200-2011, ViT-B/32 | top-1 accuracy (%): 82.2 -> 80.2 | Hurt |
| increase prototypes per class from 10 to 20 (ablation) | PRISM · CUB-200-2011, ViT-B/16 | top-1 accuracy (%): 87.8 -> 88.5 | Improved |
| increase prototype dimension from 160 to 200 (ablation) | PRISM · CUB-200-2011, ViT-B/16 | top-1 accuracy (%): 87.8 -> 88.7 | Improved |
| add CLIP-based semantic guidance to ProtoPNet prototype learning | ProtoPNet · CUB-200-2011 and Stanford Dogs | top-1 accuracy: +2.3% to +4.6% on CUB, +1.9% to +3.4% on Dogs | Improved |
| use PRISM instead of concept bottleneck models using CLIP and GPT-3 concepts | Label-Free CBM and FVLC · CUB-200-2011, ViT-B/16 | accuracy (%): 76.4 -> 87.8 and 77.9 -> 87.8 | Improved |
| add PRISM prototype alignment | ProtoPNet · CUB-200-2011, ViT-B/16 | prototype consistency (%): 19.85 -> 75.75 | Improved |
Closest papers by meaning
- Interpretable Image Classification via Non-parametric Part Prototype Learning (CVPR 2025 · similarity 0.91)
- Interpretable Image Classification with Adaptive Prototype-based Vision Transformers (NeurIPS 2024 · similarity 0.89)
- ProtoArgNet: Interpretable Image Classification with Super-Prototypes and Argumentation (AAAI 2025 · similarity 0.88)
- Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.86)
- Improving Prototypical Visual Explanations with Reward Reweighing, Reselection, and Retraining (ICML 2024 · similarity 0.86)
- LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Vision (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
