Research map / Label-efficient and robust learning / Recognition and long-tail

How Effective Can Dropout Be in Multiple Instance Learning ?

ICML 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Partial and multi-label learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Empirically and theoretically studies dropout in multiple instance learning, finds that dropping the top-k most attended instances lowers gradient direction error and yields flatter minima, and proposes MIL-Dropout, a plug-in method that drops top-k attention instances plus similar instances selected via a query mechanism.

Problem

Dropout, the most common regularizer, has not been studied for multiple instance learning (MIL), where noisy backbone features and weak bag-level supervision hinder generalization, especially in WSI classification.

Approach

Instead of random instance dropout, drop the top-k most important instances (ranked by a non-parametric sigmoid-avgpool importance score on the shallow extractor's features) together with instances most similar to them, which lowers gradient direction error and leads to flatter minima and better robustness to noise.

Key results

• ABMIL with MIL-Dropout reaches 0.964 accuracy on MUSK1 and 0.789 on FOX versus 0.892 and 0.615 for ABMIL alone in the benchmark table.
• MIL-Dropout improves four WSI aggregators on CAMELYON16, with average gains of 2.3% accuracy, 2.46% F1, and 2.48% AUC.
• On TCGA-NSCLC, MIL-Dropout yields average gains of 2.05% accuracy, 2.44% F1, and 1.80% AUC.

Measured effects reported in the paper

Surprising findings

• Dropping the top-k most attended instances, rather than random or bottom-k instances, is the only strategy that decreases gradient direction error during training.
• Integrating MIL-Dropout into a transformer block (TransMIL) failed to converge, with accuracy dropping to 64.5%.

Limitations

• Does not account for instance uncertainty when selecting instances to drop.
• Not extended to other tasks such as video frame detection or 3D point cloud classification.
• Aggressive dropout rates cause convergence failure, and integrating into transformer blocks was unsuccessful.

Experimental setup

Datasets: MUSK1 MUSK2 FOX TIGER ELEPHANT CAMELYON16 TCGA-NSCLC

Benchmarks: MUSK1 MUSK2 FOX TIGER ELEPHANT CAMELYON16 TCGA-NSCLC

Metrics: accuracy F1 AUC gradient direction error epsilon-sharpness

Baselines: mi-Net MI-Net MI-Net with DS MI-Net with RC ABMIL-Gated GNN-MIL DP-MINN NLMIL ANLMIL DSMIL TransMIL DTFD-MIL

Builds on: ABMIL DropInstance DropNeuron PDL

Closest papers by meaning

• CaMIL: Causal Multiple Instance Learning for Whole Slide Image Classification (AAAI 2024 · similarity 0.88)
• Dynamic Policy-Driven Adaptive Multi-Instance Learning for Whole Slide Image Classification (CVPR 2024 · similarity 0.86)
• Queryable Prototype Multiple Instance Learning with Vision-Language Models for Incremental Whole Slide Image Classification (AAAI 2025 · similarity 0.85)
• ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification (CVPR 2024 · similarity 0.85)
• Boosting Multiple Instance Learning Models for Whole Slide Image Classification: A Model-Agnostic Framework Based on Counterfactual Inference (AAAI 2024 · similarity 0.85)
• Advancing Multiple Instance Learning with Continual Learning for Whole Slide Imaging (CVPR 2025 · similarity 0.84)