Research map / Detection and segmentation / CNN features
Context modeling for semantic segmentation
Capture context with structured models, prototypes, and CRFs for semantic segmentation.
241 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in CNN features, keyword cluster “local · context · semantic segmentation”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
39 papers in 2023–24 and 58 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 11.9% to 16.7%.
Open questions the data points to
- Papers report UPerNet on ADE20K (mIoU) anywhere from 38.9 to 48: comparisons against it depend on whose number you take.
- Papers report UperNet on ADE20K validation (mIoU) anywhere from 42.66 to 51.66: comparisons against it depend on whose number you take.
No longer reported by papers here: Cityscapes test (mIoU, last in 2023); ADE20K validation (mIoU, last in 2022); PASCAL VOC 2012 test (mIoU, last in 2020); ADE20K val (mIoU, last in 2023).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Attention to Scale: Scale-Aware Semantic Image Segmentation (CVPR 2016 · 1,446 citations)
- 2017 · RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation (CVPR 2017 · 3,229 citations)
- 2018 · DenseASPP for Semantic Segmentation in Street Scenes (CVPR 2018 · 1,669 citations)
- 2019 · Dual Attention Network for Scene Segmentation (CVPR 2019 · 6,963 citations)
- 2020 · Squeeze-and-Attention Networks for Semantic Segmentation (CVPR 2020 · 340 citations)
- 2021 · Rethinking BiSeNet for Real-Time Semantic Segmentation (CVPR 2021 · 786 citations)
- 2022 · UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-Wise Perspective with Transformer (AAAI 2022 · 1,093 citations)
- 2023 · PIDNet: A Real-Time Semantic Segmentation Network Inspired by PID Controllers (CVPR 2023 · 735 citations)
- 2024 · SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentation (AAAI 2024 · 178 citations)
- 2025 · SegMAN: Omni-scale Context Modeling with State Space Models and Local Attention for Semantic Segmentation (CVPR 2025 · 65 citations)
- 2026 · Revisiting Downsampling in Semantic Segmentation: Fighting Aliasing with Dynamic Gaussian and Gabor Frequency Filters (AAAI 2026 · 3 citations)
Earliest papers here
- Attention to Scale: Scale-Aware Semantic Image Segmentation (CVPR 2016 · 1,446 citations)
- Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation (CVPR 2016 · 856 citations)
- DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation (CVPR 2016 · 560 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
Cityscapes test · mIoU (higher is better)
31 papers, 2017–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 79.2 | Semantic Segmentation via Structured Patch Prediction, Context CRF and Guidance CRF (CVPR 2017 · 67 citations) |
| 2018 | 80.6 | DenseASPP for Semantic Segmentation in Street Scenes (CVPR 2018 · 1,669 citations) |
| 2019 | 83.5 | Improving Semantic Segmentation via Video Propagation and Label Relaxation (CVPR 2019 · 434 citations) |
| 2021 | 85.6 | InverseForm: A Loss Function for Structured Boundary-Aware Segmentation (CVPR 2021 · 153 citations) |
Cityscapes val · mIoU (higher is better)
22 papers, 2019–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 79.06 | Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation (CVPR 2019 · 266 citations) |
| 2020 | 82.1 | Joint Semantic Segmentation and Boundary Detection Using Iterative Pyramid Contexts (CVPR 2020 · 142 citations) |
| 2021 | 87 | InverseForm: A Loss Function for Structured Boundary-Aware Segmentation (CVPR 2021 · 153 citations) |
ADE20K validation · mIoU (higher is better)
11 papers, 2018–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 44.65 | Context Encoding for Semantic Segmentation (CVPR 2018 · 1,489 citations) |
| 2019 | 45.38 | Adaptive Pyramid Context Network for Semantic Segmentation (CVPR 2019 · 443 citations) |
| 2020 | 46.27 | Context Prior for Scene Segmentation (CVPR 2020 · 313 citations) |
| 2021 | 46.65 | Consistent-Separable Feature Representation for Semantic Segmentation (AAAI 2021 · 4 citations) |
| 2022 | 52.4 | Active Boundary Loss for Semantic Segmentation (AAAI 2022 · 56 citations) |
PASCAL VOC 2012 test · mIoU (higher is better)
11 papers, 2016–2020 · not reported since 2020
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 75.7 | Attention to Scale: Scale-Aware Semantic Image Segmentation (CVPR 2016 · 1,446 citations) |
| 2017 | 82.5 | Semantic Segmentation via Structured Patch Prediction, Context CRF and Guidance CRF (CVPR 2017 · 67 citations) |
| 2018 | 82.9 | Context Encoding for Semantic Segmentation (CVPR 2018 · 1,489 citations) |
| 2019 | 88.7 | ZigZagNet: Fusing Top-Down and Bottom-Up Context for Object Segmentation (CVPR 2019 · 73 citations) |
PASCAL VOC 2012 val · mIoU (%) (higher is better)
11 papers, 2016–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 66.35 | Semantic Image Segmentation With Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform (CVPR 2016 · 392 citations) |
| 2018 | 73.22 | DifNet: Semantic Segmentation by Diffusion Networks (NeurIPS 2018 · 22 citations) |
| 2019 | 86 | ZigZagNet: Fusing Top-Down and Bottom-Up Context for Object Segmentation (CVPR 2019 · 73 citations) |
ADE20K val · mIoU (higher is better)
10 papers, 2019–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 44.89 | Co-Occurrent Features in Semantic Segmentation (CVPR 2019 · 322 citations) |
| 2020 | 46.37 | Dynamic Sampling Network for Semantic Segmentation (AAAI 2020 · 5 citations) |
| 2022 | 56.7 | GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models (NeurIPS 2022 · 61 citations) |
| 2023 | 58.22 | IDRNet: Intervention-Driven Relation Network for Semantic Segmentation (NeurIPS 2023 · 14 citations) |
What has been tried
202 papers here report 2,280 measured changes (a component added, removed or swapped, compared with a baseline): helped 77%, no clear effect 5%, hurt 15%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add CFBlock in place of ResBlock | Cityscapes Seg100 · mIoU(%) | 76.4 -> 78.5 (+2.1) | Helped | SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentation (AAAI 2024 · 178 citations) |
| Add FreqSelect | Dilated ResNet-50 theoretical receptive field · Receptive field | 1007 -> 1100 | Helped | Frequency-Adaptive Dilated Convolution for Semantic Segmentation (CVPR 2024 · 144 citations) |
| add an object transformer | MOSE val · J &F | 65.2 -> 67.3±0.36 | Helped | Putting the Object Back into Video Object Segmentation (CVPR 2024 · 132 citations) |
| Add the hierarchical encoder instead of the plain transformer | A-150 · mIoU | 23.7 -> 28.2 | Helped | SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation (CVPR 2024 · 81 citations) |
| replace LASS with MaxViT | ADE20K · mIoU | 51.3 -> 47.2 | Hurt | SegMAN: Omni-scale Context Modeling with State Space Models and Local Attention for Semantic Segmentation (CVPR 2025 · 65 citations) |
| replace the UNet head with MFS | TUT · F1 | 0.8260 -> 0.8390 | Helped | SCSegamba: Lightweight Structure-Aware Vision Mamba for Crack Segmentation in Structures (CVPR 2025 · 64 citations) |
| add prototype selection | Cityscapes panoptic segmentation · PQ | 48.7 -> 61.1 | Helped | PEM: Prototype-based Efficient MaskFormer for Image Segmentation (CVPR 2024 · 53 citations) |
| Add two 1×1 convolutions in Path1×1 1×1 instead of none | Cityscapes · mIoU (%) | 76.1 -> 76.7 | Helped | Golden Cudgel Network for Real-Time Semantic Segmentation (CVPR 2025 · 41 citations) |
| Add contextual contrastive learning | Cityscapes val · mIoU (%) | 78.48 -> 81.88 | Helped | Contextrast: Contextual Contrastive Learning for Semantic Segmentation (CVPR 2024 · 39 citations) |
| Add boundary sensitive refinement | Kvasir · mIoU | 82.75 -> 84.97 | Helped | Polyper: Boundary Sensitive Polyp Segmentation (AAAI 2024 · 37 citations) |
| Replace pooling token-mixer with DW Conv 7 × 7 | ISIC-2017 · Jaccard | 82.62±0.71 -> 82.55±0.57 | No clear effect | nnWNet: Rethinking the Use of Transformers in Biomedical Image Segmentation and Calling for a Unified Evaluation Benchmark (CVPR 2025 · 36 citations) |
| Add Phase II, including CI Loss, after Phase I | CrackSeg5k · F1 | 58.57 -> 60.08 | Helped | Mind Marginal Non-Crack Regions: Clustering-Inspired Representation Learning for Crack Segmentation (CVPR 2024 · 28 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| UPerNet | ADE20K | mIoU | 38.9 – 48 | 2 papers, e.g. ICLR 2024 and NeurIPS 2024 |
| UperNet | ADE20K validation | mIoU | 42.66 – 51.66 | 2 papers, e.g. CVPR 2020 and AAAI 2022 |
| UPerNet | ADE20K val | mIoU | 43.02 – 48 | 2 papers, e.g. NeurIPS 2023 and NeurIPS 2022 |
| DeepLab-v2 | Pascal VOC 2012 test | mIoU (%) | 71.6 – 79.7 | 2 papers, e.g. AAAI 2020 and CVPR 2017 |
| PSPNet | ADE20K validation | mIoU | 41.68 – 44.94 | 2 papers, e.g. CVPR 2019 and CVPR 2018 |
| DeepLabv3+ | PASCAL VOC 2012 val | mIoU (%) | 75.49 – 80 | 2 papers, e.g. AAAI 2022 and CVPR 2021 |
What reviewers will expect
Datasets and baselines named in the 28 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Cityscapes · 6 of 28 papers
- ADE20K · 4 of 28 papers
- ImageNet-1K · 2 of 28 papers
- DeepGlobe · 2 of 28 papers
- Inria Aerial · 2 of 28 papers
- ISIC · 2 of 28 papers
- CRAG · 2 of 28 papers
- CamVid · 2 of 28 papers
Baselines
- PSPNet · 3 of 28 papers
- DeepLabv3 · 3 of 28 papers
- UNet · 3 of 28 papers
- SegFormer · 3 of 28 papers
- STDC · 3 of 28 papers
- CMX · 2 of 28 papers
- CMNeXt · 2 of 28 papers
- DFormer · 2 of 28 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 97 papers; from OpenAlex and the papers' affiliations).
- Beijing Institute of Technology · 5 papers
- The University of Hong Kong · 4 papers
- The Chinese University of Hong Kong · 4 papers
- Chinese Academy of Sciences · 4 papers
- University of Chinese Academy of Sciences · 4 papers
- Zhejiang University · 3 papers
- Wuhan University · 3 papers
- Peng Cheng Laboratory · 3 papers
- Alibaba Group · 3 papers
- Shenzhen University · 3 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Local features and context in CNNs (170 papers)
- Context for hard detection conditions (133 papers)
- Domain-specific segmentation and detection (118 papers)
- Forgery detection (46 papers)
- Lane detection (32 papers)
Elsewhere in the topic
- Region-based and query-based detectors (183 papers)
- Convolution-transformer backbones (101 papers)
- Box regression and loss design (94 papers)
- Efficient DETR query design (85 papers)
- Boundary-aware instance segmentation (83 papers)
- Plain and high-resolution ViTs (69 papers)
- Arbitrary-shape scene text detection (61 papers)
- RGB salient object detection (61 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).