Research map / Vision-language and multimodal / Human and embodied modeling
Why do These Match? Explaining the Behavior of Image Similarity Models
Bryan A. Plummer, Mariya I. Vasileva, Vitali Petsiuk, Kate Saenko, David Forsyth
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RISE instead of LIME as saliency generator | SANE saliency · Polyvore Outfits | insertion AUC: 55.6 -> 61.2 | Improved |
| use RISE instead of LIME as saliency generator | SANE saliency · Animals with Attributes 2 | deletion AUC: 71.7 -> 74.9 | Hurt |
| keep the reference image fixed instead of manipulating both (ablation) | RISE · Polyvore Outfits | insertion AUC: 58.5 -> 61.2 | Improved |
| keep the reference image fixed instead of manipulating both (ablation) | RISE · Polyvore Outfits | deletion AUC: 50.6 -> 46.8 | Improved |
| add map matching (MM) to SANE (ablation) | SANE · Polyvore Outfits | insertion: 26.3 -> 27.1 | Improved |
| add map matching (MM) to SANE (ablation) | SANE · Animals with Attributes 2 | insertion: 0.1 -> 6.0 | Improved |
| add TCAV scores to SANE + MM (ablation) | SANE + MM · Polyvore Outfits | insertion: 27.1 -> 31.5 | Improved |
| add TCAV scores to SANE + MM (ablation) | SANE + MM · Animals with Attributes 2 | deletion: -10.7 -> -24.1 | Improved |
| add saliency-map supervision L_hm to attribute classifier (ablation) | Attribute Classifier · Polyvore Outfits | attribute mAP: 53.2 -> 56.0 | Improved |
| add saliency-map supervision L_hm to attribute classifier (ablation) | Attribute Classifier · Animals with Attributes 2 | attribute mAP: 65.1 -> 67.2 | Improved |
| use Huber loss instead of BCE for attribute explanation (ablation) | Attribute Classifier · Polyvore Outfits | deletion: -5.8 -> -8.1 | Improved |
| show SANE attribute explanations to users instead of no explanation | user study · Animals with Attributes 2 | user accuracy: 42.0 -> 61.3 | Improved |
Closest papers by meaning
- Rethinking Saliency Maps: A Cognitive Human Aligned Taxonomy and Evaluation Framework for Explanations (AAAI 2026 · similarity 0.87)
- Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods (CVPR 2024 · similarity 0.86)
- Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactions (NeurIPS 2025 · similarity 0.86)
- I-CEE: Tailoring Explanations of Image Classification Models to User Expertise (AAAI 2024 · similarity 0.86)
- Explainable Saliency: Articulating Reasoning with Contextual Prioritization (CVPR 2025 · similarity 0.86)
- Language Model as Visual Explainer (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
