Research map / Label-efficient and robust learning / Recognition and long-tail
Take the Bull by the Horns: Learning to Segment Hard Samples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes L2S, a medical image segmentation framework that identifies hard samples via a Bayesian-optimized Dice threshold and a classifier, then segments them with a diffusion-based feature augmentation and a Mask2Former-style decoder.
Problem
Medical image segmentation accuracy is poor on hard samples (e.g., pathological variations, artifacts, rare cases), which existing end-to-end and coarse-to-fine methods handle poorly.
Approach
Separates normal and hard samples using a learned hardness classifier with a dynamically tuned Dice threshold, then processes hard samples with a dedicated Mask2Former-based network whose diffusion-generated noisy label features act as implicit second-order (Hessian-trace) regularization.
Key results
- On ISIC2018, L2S improves Dice on hard samples by 8.97% over SOTA methods and overall Dice by 1.01% (abstract).
- L2S achieves average Dice 87.21% and IoU 80.07% across four binary segmentation datasets, versus 86.48% Dice for the best baseline (TransFuse).
- On ACDC, L2S (Mask2former backbone) reaches average Dice 91.68%, versus 91.03% for nnUNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FFT-based feature alignment (ablation) | L2S · ISIC 2018 | DSC: 90.66 (-0.3) -> 90.97 | Helped |
| Add consistency loss (ablation) | L2S · ISIC 2018 | DSC: 90.78 (-0.2) -> 90.97 | Helped |
| Add adaptive threshold selection (ablation) | L2S · ISIC 2018 | DSC: 90.86 (-0.1) -> 90.97 | Helped |
| Add FFT, consistency loss, and adaptive threshold selection (ablation) | L2S · ISIC 2018 | DSC: 90.54 (-0.4) -> 90.97 | Helped |
| Distinguish normal and hard samples with the pretrained classifier (ablation) | L2S · ISIC 2018 | DSC: 89.59 -> 90.97 | Helped |
| Add diffusion feature augmentation (ablation) | L2S · ISIC 2018 | DSC†: 59.38 -> 63.90 | Helped |
| Add the Post-Former module (ablation) | L2S · ISIC 2018 | DSC: 90.36 -> 90.97 | Helped |
| Use multiscale features instead of single-scale features (ablation) | L2S hard-sample segmentation · ISIC 2018 | DSC†: 62.24 -> 63.90 | Helped |
| Add learnable query features (ablation) | L2S hard-sample segmentation · ISIC 2018 | DSC†: 60.82 -> 63.90 | Helped |
| Use L2S with Mask2Former rather than the UNet backbone (ablation) | L2S · ACDC | Average Dice: 91.36 -> 91.68 | Helped |
Surprising findings
- Applying a similar diffusion procedure to normal samples yields minimal improvement, suggesting different optimization strategies are needed for easy and hard samples.
- Removing the hard-sample identification mechanism lowers Dice from 90.87% to 89.59%, indicating that extra processing can hurt normal samples.
Limitations
- Classifier accuracy (about 83% on ISIC2018, 79% on BUSI) limits hard-sample identification and overall gains.
- Theoretical justification relies on Taylor approximation and zero-mean/center-distribution assumptions.
- Reported results are from reproduced baselines with five-fold cross-validation; significance details deferred to supplementary material.
Experimental setup
Datasets: COSMOS2022 ISIC2018 Kvasir-SEG BUSI BTCV ACDC
Benchmarks: COSMOS2022 ISIC2018 Kvasir-SEG BUSI BTCV ACDC
Metrics: Dice score IoU Dice on hard samples (DSC†)
Baselines: UNet nnUNet Polyp-PVT MedT TransUNet TransFuse SwinUNet MedSegDiffV2 Mask2Former
Builds on: Mask2Former MedSegDiffV2 Hessian regularization diffusion probabilistic model
Closest papers by meaning
- Learning Hierarchical Image Segmentation For Recognition and By Recognition (ICLR 2024 · similarity 0.80)
- POT: Prototypical Optimal Transport for Weakly Supervised Semantic Segmentation (CVPR 2025 · similarity 0.80)
- Exploring Learning Complexity for Efficient Downstream Dataset Pruning (ICLR 2025 · similarity 0.79)
- Nearly Optimal Sample Complexity for Learning with Label Proportions (ICML 2025 · similarity 0.79)
- Scaling up Image Segmentation across Data and Tasks (CVPR 2025 · similarity 0.79)
- A Generative Approach at the Instance-Level for Image Segmentation Under Limited Training Data Conditions (Student Abstract) (AAAI 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.