Research map / Label-efficient and robust learning / Recognition and long-tail

SI-MIL: Taming Deep MIL for Self-Interpretability in Gigapixel Histopathology

CVPR 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Prototype and concept dictionary models · 15 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SI-MIL, a dual-branch MIL that co-trains a deep ABMIL branch with a linear, self-interpretable branch over handcrafted nuclei-based PathExpert features to give feature-level explanations for WSI predictions.

Problem

Whole slide image classification with MIL models is typically black-box; existing interpretability only highlights salient patches and post-hoc feature analyses disconnect from the deep model's computations, so pathologists get little feature-level rationale.

Approach

Co-learn a deep additive ABMIL branch and a linear predictor on handcrafted PathExpert nuclei features, linked by a differentiable perturbed Top-K patch selection guided by MIL attention and a knowledge distillation loss, so the final prediction is a linear function of interpretable features.

Key results

• SI-MIL reaches 0.944 accuracy and 0.968 AUC on TCGA-BRCA versus 0.937/0.974 for DINO ViT-S ABMIL, with a linear, self-interpretable prediction.
• On TCGA-Lung SI-MIL gets 0.884 accuracy and 0.941 AUC, above the PathFeat baseline (0.830/0.888) and the 2-stage training baseline (0.865/0.932).
• A pathologist judged 90% (IDC) and 80% (ILC) of top-10 features relevant; in the faithfulness study 44.5% of features were highly relevant, 28.3% moderately, and 27.2% non-relevant.

Measured effects reported in the paper

Surprising findings

• Training a MIL directly on PathExpert features without the projector drops performance substantially (e.g., 0.767 accuracy on TCGA-Lung), and PathFeat-style training underperforms deep features.
• Replacing the perturbed Top-K with a non-differentiable selection degrades performance, suggesting regions discriminative in deep space are less discriminative in PathExpert space.

Limitations

• Evaluated only with DINO ViT-S deep features; other foundation models left for future work.
• Interpretability validated with a single pathologist on a small set of slides; some features judged non-relevant and possibly affected by HoVer-Net misclassifications.
• Classification evaluated on three TCGA cohorts with binary tasks only.

Experimental setup

Datasets: TCGA-BRCA TCGA-Lung TCGA-CRC PanNuke

Benchmarks: TCGA-BRCA TCGA-Lung TCGA-CRC

Metrics: accuracy AUC Silhouette score JS divergence user-friendliness faithfulness

Baselines: Additive ABMIL with IN ViT-S RetCCL CTransPath DINO ViT-S PathFeat 2-stage training CLAM TransMIL

Builds on: ABMIL CLAM TransMIL DINO HoVer-Net perturbed top-k

Closest papers by meaning

• Promptable Representation Distribution Learning and Data Augmentation for Gigapixel Histopathology WSI Analysis (AAAI 2025 · similarity 0.89)
• Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology (NeurIPS 2026 · similarity 0.89)
• xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology (NeurIPS 2024 · similarity 0.89)
• CaMIL: Causal Multiple Instance Learning for Whole Slide Image Classification (AAAI 2024 · similarity 0.87)
• Do Multiple Instance Learning Models Transfer? (ICML 2025 · similarity 0.87)
• PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.86)