Research map / Detection and segmentation / CNN features
Contextrast: Contextual Contrastive Learning for Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Contextrast, a supervised contrastive learning framework for semantic segmentation that combines multi-scale contextual contrastive learning (CCL, with anchors updated from the last layer) and boundary-aware negative (BANE) sampling, improving DeepLabV3, HRNet and OCRNet without extra inference cost.
Problem
Semantic segmentation networks lack local/global context and fail to model relationships between them, producing under- and over-segmentation, especially along object boundaries.
Approach
Representative class anchors are computed at each encoder layer and updated by aggregating the last-layer anchors, so lower-layer anchors carry both local and global context; negatives are sampled from the boundaries of incorrectly predicted regions, as harder negatives for an InfoNCE-style pixel-anchor loss.
Key results
- HRNet on Cityscapes val improves from 78.48 to 82.20 mIoU with Contextrast (+3.72), versus 81.50 for multi/cross-scale contrastive learning.
- DeepLabV3 on ADE20K improves from 42.85 to 44.12 mIoU (+1.27) and on CamVid from 78.80 to 79.98 (+1.18).
- On Cityscapes test, HRNet + Contextrast reaches 82.14 iIoU-category-level with gains of +3.10 on instance-level iIoU metrics for the class category.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CCL and BANE sampling | DeepLabV3 · Cityscapes val | mIoU (%): 77.12 -> 79.35 | Helped |
| Add CCL and BANE sampling | HRNet · CamVid | mIoU (%): 82.17 -> 84.33 | Helped |
| Add CCL and BANE sampling | OCRNet · ADE20K | mIoU (%): 41.51 -> 43.84 | Helped |
| Add CCL and BANE sampling | DeepLabV3 · COCO-Stuff | mIoU (%): 37.92 -> 38.12 | Helped |
| Add CCL and BANE sampling | OCRNet · COCO-Stuff | mIoU (%): 39.00 -> 39.08 | Helped |
| Add CCL and BANE sampling | DeepLabV3 · PASCAL-C | mIoU (%): 52.01 -> 52.62 | Helped |
| Add contextual contrastive learning (ablation) | HRNet + BANE · Cityscapes val | mIoU (%): 78.48 -> 81.88 | Helped |
| Add semi-hard sampling (ablation) | HRNet + CCL · Cityscapes val | mIoU (%): 81.88 -> 82.01 | Helped |
| Use BANE instead of semi-hard sampling (ablation) | HRNet + CCL · Cityscapes val | mIoU (%): 82.01 -> 82.20 | Helped |
| Add contextual contrastive learning (ablation) | HRNet + BANE · CamVid | mIoU (%): 82.17 -> 83.14 | Helped |
| Add semi-hard sampling (ablation) | HRNet + CCL · CamVid | mIoU (%): 83.14 -> 83.38 | Helped |
| Use BANE instead of semi-hard sampling (ablation) | HRNet + CCL · CamVid | mIoU (%): 83.38 -> 84.33 | Helped |
Surprising findings
- Multi/cross-scale contrastive learning from prior work sometimes degrades performance (e.g., OCRNet on COCO-Stuff, DeepLabV3 on PASCAL-C), attributed to conflicting loss terms.
- Sampling 100% of error-region features as negatives gives lower mIoU (81.10) than the 50% ratio (82.20).
Limitations
- Gains on some settings are small (e.g., +0.30 on COCO-Stuff for HRNet, +0.08 on COCO-Stuff for OCRNet).
- Hyperparameters (anchor fusion weights, sampling ratio K, loss weights) are tuned on validation sets and the best setting depends on the dataset.
Experimental setup
Datasets: Cityscapes CamVid COCO-Stuff ADE20K PASCAL-C ImageNet
Benchmarks: Cityscapes CamVid COCO-Stuff ADE20K PASCAL-C
Metrics: mIoU iIoU boundary mIoU alignment uniformity neighborhood uniformity
Baselines: ContrastiveSeg Region-aware contrastive learning Multi/cross-scale contrastive learning (Pissas et al.) semi-hard negative sampling HRNet OCRNet DeepLabV3
Builds on: ContrastiveSeg region-aware contrastive learning multi-scale and cross-scale contrastive learning OCRNet HRNet
Closest papers by meaning
- Context-Reinforced Semantic Segmentation (CVPR 2019 · similarity 0.89)
- Context Prior for Scene Segmentation (CVPR 2020 · similarity 0.88)
- Context Encoding for Semantic Segmentation (CVPR 2018 · similarity 0.88)
- Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation (CVPR 2022 · similarity 0.87)
- Bootstrapping Semantic Segmentation with Regional Contrast (ICLR 2022 · similarity 0.86)
- Looking Beyond Single Images for Contrastive Semantic Segmentation Learning (NeurIPS 2021 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.