Research map / Label-efficient and robust learning / Recognition and long-tail
CAPE: CAM as a Probabilistic Ensemble for Enhanced DNN Interpretation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reformulates CAM and the softmax classification layer as a probabilistic ensemble (CAPE) whose per-region contributions sum to the image-level prediction, plus a mutual-region variant (μ-CAPE), trained via bootstrap distillation.
Problem
CAM heatmaps only give relative, class-wise attention: they are not comparable across classes and do not reveal how much each image region contributes to the model's class prediction.
Approach
Replace the softmax output layer with a probabilistic ensemble over spatial locations, using a softmax over spatial dimensions as region saliency, so that the per-voxel contributions exactly decompose the image-level prediction; μ-CAPE restores class-mutual regions by using a single-softmax form, and bootstrap distillation with a learnable temperature trains the layer.
Key results
- On CUB with ResNet-50, μ-CAPE (TS) reaches AD 10.3 (lower is better) and IC 48.5, versus 21.2 AD and 27.9 IC for CAM.
- CAPE (PF) reduces mIoU on ImageNet to 11.0 and on CMML to 4.9 versus 84.4 and 0.1 for CAM... (mIoU rows compared per dataset in Table 1; CAPE is designed for class-discriminative regions with lower overlap).
- Bootstrap-trained CAPE on ImageNet with ResNet-50 achieves 74.64% accuracy versus 76.13% for the vanilla classifier, showing a small accuracy gap.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CAPE explanation with μ-CAPE explanation (ablation) | CAPE (PF), ResNet-50 · CUB200-2011 | mIoU: 13.4 -> 66.6 | Hurt |
| Replace CAPE explanation with μ-CAPE explanation (ablation) | CAPE (PF), ResNet-50 · CUB200-2011 | BC: 3 -> 5 | Helped |
| Replace CAPE explanation with μ-CAPE explanation (ablation) | CAPE (PF), ResNet-50 · ImageNet ILSVRC2012 | mIoU: 11.0 -> 70.3 | Hurt |
| Replace CAPE explanation with μ-CAPE explanation (ablation) | CAPE (PF), ResNet-50 · ImageNet ILSVRC2012 | BC: 7 -> 5 | No clear effect |
| Replace CAPE explanation with μ-CAPE explanation (ablation) | CAPE (PF), ResNet-50 · CMML | mIoU: 4.9 -> 0.6 | Hurt |
| Replace CAPE explanation with μ-CAPE explanation (ablation) | CAPE (PF), ResNet-50 · CMML | BC: 0 -> 7 | Helped |
| Train μ-CAPE from scratch rather than post-fitting (ablation) | μ-CAPE, ResNet-50 · CUB200-2011 | BC: 5 -> 12 | Helped |
| Train μ-CAPE from scratch rather than post-fitting (ablation) | μ-CAPE, ResNet-50 · ImageNet ILSVRC2012 | BC: 5 -> 9 | Helped |
| Train μ-CAPE from scratch rather than post-fitting (ablation) | μ-CAPE, ResNet-50 · CMML | BC: 7 -> 9 | Helped |
| Bootstrap-train CAPE instead of using off-the-shelf CAPE parameters (ablation) | CAPE, ResNet-50 · CMML | accuracy: 87.4 -> 90.3 (PF); 87.4 -> 89.8 (TS) | Helped |
| Bootstrap-train CAPE instead of using off-the-shelf CAPE parameters (ablation) | CAPE, ResNet-50 · CUB200-2011 | accuracy: 80.62 -> 82.12 (PF); 80.62 -> 82.19 (TS) | Helped |
| Bootstrap-train CAPE instead of using off-the-shelf CAPE parameters (ablation) | CAPE, ResNet-50 · ImageNet ILSVRC2012 | accuracy: 74.01 -> 74.42 (PF); 74.01 -> 74.64 (TS) | Helped |
Surprising findings
- CAPE explanations do not faithfully explain the vanilla classifier: prediction disagreement between CAPE and the vanilla layer is non-negligible, so CAPE should not be used to explain the vanilla classifier's decisions.
- Naive averaging and off-the-shelf CAPE lose classification accuracy relative to the vanilla layer, and direct cross-entropy training of CAPE does not improve over off-the-shelf CAPE; bootstrap distillation is needed.
Limitations
- Training convergence issues caused by soft prediction confidence over the large H×W×|C| softmax denominator, leading to a small accuracy gap versus the vanilla classifier.
- CAPE predictions disagree non-negligibly with the vanilla classifier, limiting its use as an explanation of the vanilla model.
- Evaluated only on ResNet-50 in the main text, with Swin Transformer results in the supplement; CMML is a small two-class dataset.
Experimental setup
Datasets: CUB-200-2011 ImageNet ILSVRC2012 CMML cytology dataset
Benchmarks: CUB-200-2011 ImageNet ILSVRC2012 CMML
Metrics: Average Drop (AD) Average Increase in Confidence (IC) Average Drop in Deletion (ADD) ADCC mean IoU Borda Count classification accuracy
Baselines: CAM Grad-CAM Grad-CAM++ Smooth Grad-CAM++ Layer-CAM Score-CAM FD-CAM LIFT-CAM
Builds on: Class Activation Maps (CAM) Score-CAM LIFT-CAM Knowledge distillation
Closest papers by meaning
- DD-CAM: Minimal Sufficient Explanations for Vision Models Using Delta Debugging (NeurIPS 2026 · similarity 0.89)
- Finer-CAM: Spotting the Difference Reveals Finer Details for Visual Explanation (CVPR 2025 · similarity 0.88)
- A Novel Visual Interpretability for Deep Neural Networks by Optimizing Activation Maps with Perturbation (AAAI 2021 · similarity 0.87)
- DiffCAM: Data-Driven Saliency Maps by Capturing Feature Differences (CVPR 2025 · similarity 0.86)
- Extracting Class Activation Maps From Non-Discriminative Features As Well (CVPR 2023 · similarity 0.86)
- Empowering CAM-Based Methods with Capability to Generate Fine-Grained and High-Faithfulness Explanations (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.