Research map / Detection and segmentation / Open-set and few-shot detection
Adaptive Sensitivity Analysis for Robust Augmentation against Natural Corruptions in Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an online, adaptive sensitivity-analysis-guided augmentation method for segmentation that samples augmentation intensities by their impact on accuracy, improving robustness to natural corruptions such as adverse weather.
Problem
Segmentation models degrade under natural corruptions (weather, noise, blur) that are hard to predict or simulate, and existing sensitivity-analysis-guided augmentation is too computationally and storage expensive for practical training use.
Approach
Replace uniform sampling over the augmentation parameter space with an adaptive, online sensitivity analysis that uses KID as an image-degradation measure and PCHIP interpolation to pick intensities at equal spacing along a sensitivity curve, then samples augmentations with a Beta-Binomial policy weighted by accuracy, avoiding offline augmented datasets.
Key results
- On zero-shot ACDC snow, the method reaches 48.16 mIoU vs 44.35-ish for next best, a reported 6.20% relative mIoU gain over the next-best method.
- Sensitivity analysis runs about 9.6 min vs 90.0 min for AdvSteer (roughly 10x faster) and uses 12 GB vs 2.4 TB storage (about 200x less).
- On Cityscapes clean evaluation with SegFormer, the method reaches 76.33 mIoU vs 75.13 for TrivialAugment and 65.32 baseline under basis perturbations (basis aug mIoU 71.67 vs 65.32 baseline).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use sensitivity-guided augmentation instead of the strongest reported baseline | SegFormer · ACDC Fog | mIoU: 60.79 -> 62.50 | Helped |
| Use sensitivity-guided augmentation instead of the strongest reported baseline | SegFormer · ACDC Rain | mIoU: 47.53 -> 49.36 | Helped |
| Use sensitivity-guided augmentation instead of the strongest reported baseline | SegFormer · ACDC Night | mIoU: 20.65 -> 20.72 | Helped |
| Use sensitivity-guided augmentation instead of the strongest reported baseline | SegFormer · ACDC Snow | mIoU: 45.35 -> 48.16 | Helped |
| Use sensitivity-guided augmentation instead of the strongest reported baseline | SegFormer · IDD zero-shot | mIoU: 40.74 -> 40.33 | Hurt |
| Use sensitivity-guided augmentation rather than no augmentation | SegFormer · ACDC zero-shot | mIoU: 35.48 -> 45.45 | Helped |
| Add geometric augmentations to photometric-only training (ablation) | Ours · ACDC ablation | mIoU: 43.69 -> 44.67 | Helped |
| Add photometric augmentations to geometric-only training (ablation) | Ours · ACDC ablation | mIoU: 37.97 -> 44.67 | Helped |
| Remove clean training warm-up (ablation) | Ours · ACDC ablation | mIoU: 44.67 -> 44.66 | No clear effect |
| Replace informed probability sampling with uniform sampling (ablation) | Ours · ACDC ablation | mIoU: 44.67 -> 43.17 | Hurt |
| Use adaptive sensitivity analysis instead of AdvSteer sensitivity analysis | Sensitivity analysis · Sensitivity analysis runtime | time: 90.0±15.5 min -> 9.6±0.2 min | Helped |
Surprising findings
- Hue channel sensitivity curves were the most volatile during training, suggesting hue is a significant adversarial factor.
- Uniform sampling of computed intensities and training without clean warm-up gave only small differences, and AugMix gave higher aAcc on rainy weather despite lower mIoU.
Limitations
- Sensitivity analysis still adds evaluation overhead during training, and some benchmarks (AdvSteer) show weaker performance for Cityscapes and Synapse.
- Night scenario performance in ACDC is worse in aAcc than some baselines.
- Generalization to other tasks and architectures is claimed but only preliminarily tested with classification on CUB.
Experimental setup
Datasets: Cityscapes ADE20K VOC2012 POTSDAM A2I2Haze Synapse ACDC IDD ImageNet CUB
Benchmarks: ACDC IDD ImageNet-C AdvSteer benchmark ADE20K VOC2012 POTSDAM A2I2Haze Cityscapes Synapse CUB
Metrics: aAcc mIoU mAcc
Baselines: AugMix AutoAugment RandAugment TrivialAugment VIPAug IDBH AdvSteer
Builds on: AdvSteer AugMix TrivialAugment IDBH
Closest papers by meaning
- Improving robustness against common corruptions by covariate shift adaptation (NeurIPS 2020 · similarity 0.88)
- On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness (NeurIPS 2021 · similarity 0.88)
- Uncertainty-aware Fine-tuning of Segmentation Foundation Models (NeurIPS 2024 · similarity 0.84)
- RobustSAM: Segment Anything Robustly on Degraded Images (CVPR 2024 · similarity 0.84)
- Pixel-level Certified Explanations via Randomized Smoothing (ICML 2025 · similarity 0.84)
- Benchmarking Segmentation Models with Mask-Preserved Attribute Editing (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.