Research map / Label-efficient and robust learning / Recognition and long-tail

From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection

ICML 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Self-explaining and visual explanation networks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes P2P, an instance-wise feature selection method that learns binary masks over superpixel regions (instead of pixels) with a logit-normal part-relationship model and a dynamic, certainty-based sparsity threshold for inherently interpretable image classification.

Problem

Instance-wise feature selection on images usually sparsifies at the pixel level, which yields uninformative, evenly spaced masks that keep predictive accuracy without giving meaningful explanations; existing methods also use a fixed sparsity level.

Approach

Select binary masks over instance-specific SLIC superpixel regions rather than pixels, model pairwise part relationships through a non-diagonal logit-normal distribution with dot-product covariance, and choose each image's sparsity level at inference by increasing the masked fraction until the classifier reaches a certainty threshold delta.

Key results

• On COCO-10 (40% sparsity), P2P reaches 89.53% accuracy versus 88.43% for COMET and 89.36% for the Blackbox, and localization of 47.01% versus 36.43% for COMET.
• On ImageNet-9 (30%), P2P achieves 69.25% localization versus 63.73% for COMET, with accuracy 94.42% close to the Blackbox 94.45%.
• On ImageNet, P2P reaches 68.70% accuracy, below COMET's 70.90% and the Blackbox's 71.22%.

Measured effects reported in the paper

Surprising findings

• Blackbox Pixel, which masks 60-80% of pixels evenly, performs close to the full-image Blackbox, suggesting pixel-level sparsity objectives do not force meaningful selection.
• COMET^-1 (inverted mask) performs about as well as COMET, which the authors take as evidence that COMET's predictions do not depend on its highlighted regions.

Limitations

• Accuracy on ImageNet is lower than COMET and the Blackbox (68.70 vs 70.90 and 71.22).
• Superpixel count (100 segments) and region proposer chosen by visual inspection; the proposer is frozen and not learned.
• Faithfulness is evaluated via insertion/deletion fidelity, and localization relies on ground-truth segmentations that exist only for some datasets.

Experimental setup

Datasets: CIFAR-10 ImageNet ImageNet-9 COCO-10 BAM Object BAM Scene

Benchmarks: CIFAR-10 COCO-10 ImageNet ImageNet-9 BAM Object BAM Scene

Metrics: accuracy localization overlap insertion fidelity deletion fidelity

Baselines: Blackbox Blackbox Pixel COMET COMET^-1 DiET RB-AEM REAL-X B-cos InfoMask

Builds on: REAL-X DiET COMET RB-AEM Group Lasso

Closest papers by meaning

• Interpretable Image Classification via Non-parametric Part Prototype Learning (CVPR 2025 · similarity 0.85)
• An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels (ICLR 2025 · similarity 0.84)
• Discover and Mitigate Multiple Biased Subgroups in Image Classifiers (CVPR 2024 · similarity 0.84)
• A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysis (ICLR 2024 · similarity 0.84)
• Measuring Per-Unit Interpretability at Scale Without Humans (NeurIPS 2024 · similarity 0.84)
• Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.84)