Research map / Label-efficient and robust learning / Recognition and long-tail
Sm: enhanced localization in Multiple Instance Learning for medical imaging classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Sm, a Dirichlet-energy-based smooth operator that models local dependencies among instances in MIL, used alone (SmAP) or with transformers (SmTAP), improving instance localization while staying competitive in bag classification.
Problem
Multiple Instance Learning for medical imaging is judged mainly on bag classification; instance-level localization of recent dependency-modeling MIL methods is comparatively poor, and local neighboring-instance dependencies are not explicitly modeled at the instance level.
Approach
Introduce a smooth operator Sm, derived as the solution of a Dirichlet-energy-regularized objective (iterative approximation with T=10 steps, normalized Laplacian, trainable alpha), applied to attention-pooling embeddings or attention values to encourage neighboring instances to receive similar attention; combined optionally with transformer encoders for global dependencies.
Key results
- On localization, the proposed SmTAP ranks first on RSNA, PANDA and CAMELYON16 instance-level AUROC/F1 (e.g., CAMELYON16 instance AUROC 0.789 vs. 0.781 for TransMIL and 0.742 for CAMIL), with an average rank of 1.5 on localization.
- On bag classification, SmAP and SmTAP achieve the best aggregated mean rank (1.833), with SmTAP reaching CAMELYON16 bag F1 0.948 and RSNA bag F1 0.825.
- Adding Sm on top of DSMIL improves CAMELYON16 instance AUROC from 0.76 to 0.96 using ResNet50-BT features.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Sm to ABMIL attention pooling (ablation) | ABMIL · RSNA | AUROC (localization): 0.806 -> 0.798 | Hurt |
| Add Sm to ABMIL attention pooling (ablation) | ABMIL · PANDA and CAMELYON16 | AUROC (localization): PANDA: 0.768 -> 0.799; CAMELYON16: 0.819 -> 0.960 | Helped |
| Add Sm to Transformer + ABMIL (ablation) | Transformer + ABMIL · PANDA and CAMELYON16 | AUROC (localization): PANDA: 0.751 -> 0.790; CAMELYON16: 0.781 -> 0.789 | Helped |
| Use Sm in attention pooling only, rather than after the transformer and in pooling (ablation) | SmTAP · CAMELYON16 | AUROC (localization): 0.841 -> 0.915 | Helped |
| Use Sm in attention pooling only, rather than after the transformer and in pooling (ablation) | SmTAP · PANDA | AUROC (localization): 0.813 -> 0.787 | Hurt |
| Use Sm at early, middle, or late attention-pooling placement (ablation) | SmAP · RSNA, PANDA, and CAMELYON16 | AUROC (localization): RSNA: 0.798, 0.806, 0.811; PANDA: 0.799, 0.792, 0.802; CAMELYON16: 0.960, 0.922, 0.819 | Mixed |
| Increase smoothing strength by setting alpha above zero (ablation) | SmAP · CAMELYON16 | Localization and classification performance | Helped |
| Use lower alpha instead of higher alpha (ablation) | SmAP · CAMELYON16 | AUROC (localization) | Helped |
| Use higher alpha instead of lower alpha (ablation) | SmAP · CAMELYON16 | AUROC (classification) | Helped |
| Add spectral normalization alongside Sm (ablation) | SmAP · CAMELYON16 | Instance-level performance | Helped |
| Increase approximation steps T above zero (ablation) | SmAP · RSNA and CAMELYON16 | AUROC (localization) | Helped |
| Apply Sm to DSMIL (ablation) | DSMIL · CAMELYON16 | Instance AUROC: 0.76 -> 0.960 | Helped |
Surprising findings
- Using a penalty term on attention smoothness instead of the operator gives comparable results, with the operator only slightly superior.
- On RSNA, the baseline ABMIL outperforms the proposed smooth attention pooling at both localization and classification, attributed to simpler CT bag graphs.
Limitations
- Computational cost of the smooth operator scales with bag size, mitigated only by possible subgraph sampling.
- No definitive answer on the best placement of Sm; optimal location may be problem-dependent.
- Evaluated only on two medical problems; clinical deployment needs wider evaluation and uncertainty quantification.
Experimental setup
Datasets: RSNA PANDA CAMELYON16
Benchmarks: RSNA PANDA CAMELYON16
Metrics: AUROC F1 average rank normalized Dirichlet energy
Baselines: ABMIL CLAM DSMIL DFTD-MIL TransMIL SETMIL GTP CAMIL
Builds on: ABMIL TransMIL spectral normalization Dirichlet energy over graphs
Closest papers by meaning
- Retrieval-Augmented Multiple Instance Learning (NeurIPS 2023 · similarity 0.87)
- Do Multiple Instance Learning Models Transfer? (ICML 2025 · similarity 0.84)
- How Effective Can Dropout Be in Multiple Instance Learning ? (ICML 2025 · similarity 0.84)
- TinyMIG: Transferring Generalization from Vision Foundation Models to Single-Domain Medical Imaging (ICML 2025 · similarity 0.84)
- Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation (CVPR 2024 · similarity 0.84)
- Are Multiple Instance Learning Algorithms Learnable for Instances? (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.