Research map / Detection and segmentation / CNN features
Unified panoptic segmentation
Perform panoptic segmentation with unified training, differentiable optimization, and relational context.
33 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2025) · an approach in CNN features, keyword cluster “instance · panoptic · mask”
State History Benchmarks Tried Baselines Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
4 papers in 2023–24 and 1 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 1.2% to 0.3%.
No longer reported by papers here: COCO val (PQ, last in 2022); Cityscapes val (PQ, last in 2023); COCO test-dev (PQ, last in 2022); ADE20K val (PQ, last in 2023); Mapillary Vistas val (PQ, last in 2021).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · Path Aggregation Network for Instance Segmentation (CVPR 2018 · 9,292 citations)
- 2019 · Panoptic Feature Pyramid Networks (CVPR 2019 · 1,411 citations)
- 2020 · Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (CVPR 2020 · 631 citations)
- 2021 · MaX-DeepLab: End-to-End Panoptic Segmentation With Mask Transformers (CVPR 2021 · 501 citations)
- 2022 · Panoptic SegFormer: Delving Deeper Into Panoptic Segmentation With Transformers (CVPR 2022 · 156 citations)
- 2023 · OneFormer: One Transformer To Rule Universal Image Segmentation (CVPR 2023 · 490 citations)
- 2025 · EOV-Seg: Efficient Open-Vocabulary Panoptic Segmentation (AAAI 2025 · 6 citations)
Earliest papers here
- Path Aggregation Network for Instance Segmentation (CVPR 2018 · 9,292 citations)
- Panoptic Feature Pyramid Networks (CVPR 2019 · 1,411 citations)
- UPSNet: A Unified Panoptic Segmentation Network (CVPR 2019 · 421 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
COCO val · PQ (higher is better)
16 papers, 2020–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 45.7 | Learning Instance Occlusion for Panoptic Segmentation (CVPR 2020 · 74 citations) |
| 2021 | 51.1 | MaX-DeepLab: End-to-End Panoptic Segmentation With Mask Transformers (CVPR 2021 · 501 citations) |
| 2022 | 55.8 | Panoptic SegFormer: Delving Deeper Into Panoptic Segmentation With Transformers (CVPR 2022 · 156 citations) |
Cityscapes val · PQ (higher is better)
16 papers, 2019–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 61.8 | UPSNet: A Unified Panoptic Segmentation Network (CVPR 2019 · 421 citations) |
| 2020 | 67 | Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (CVPR 2020 · 631 citations) |
| 2023 | 67.2 | OneFormer: One Transformer To Rule Universal Image Segmentation (CVPR 2023 · 490 citations) |
COCO test-dev · PQ (higher is better)
13 papers, 2019–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 46.6 | UPSNet: A Unified Panoptic Segmentation Network (CVPR 2019 · 421 citations) |
| 2020 | 47.8 | SOGNet: Scene Overlap Graph Network for Panoptic Segmentation (AAAI 2020 · 60 citations) |
| 2021 | 51.5 | REFINE: Prediction Fusion Network for Panoptic Segmentation (AAAI 2021 · 10 citations) |
| 2022 | 56.2 | Panoptic SegFormer: Delving Deeper Into Panoptic Segmentation With Transformers (CVPR 2022 · 156 citations) |
ADE20K val · PQ (higher is better)
5 papers, 2020–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 32.4 | Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation (NeurIPS 2020 · 19 citations) |
| 2022 | 37 | Panoptic, Instance and Semantic Relations: A Relational Context Encoder To Enhance Panoptic Segmentation (CVPR 2022 · 19 citations) |
| 2023 | 49.8 | OneFormer: One Transformer To Rule Universal Image Segmentation (CVPR 2023 · 490 citations) |
Mapillary Vistas val · PQ (higher is better)
5 papers, 2019–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 39 | Seamless Scene Segmentation (CVPR 2019 · 228 citations) |
| 2020 | 40.3 | Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (CVPR 2020 · 631 citations) |
| 2021 | 45.1 | Improving Panoptic Segmentation at All Scales (CVPR 2021 · 21 citations) |
What has been tried
32 papers here report 386 measured changes (a component added, removed or swapped, compared with a baseline): helped 75%, no clear effect 6%, hurt 17%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use ViT-based CLIP block 0 instead of block 5 or 11 as spatial awareness extractor | ADE20K · PQ | 23.8 -> 24.5; 23.7 -> 24.5 | Helped | EOV-Seg: Efficient Open-Vocabulary Panoptic Segmentation (AAAI 2025 · 6 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Seamless | Cityscapes val | PQ (%) | 60.3 – 65 | 2 papers, e.g. CVPR 2020 and CVPR 2020 |
| Panoptic-DeepLab | Cityscapes val | PQ | 59.9 – 63 | 2 papers, e.g. CVPR 2022 and CVPR 2022 |
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 5 papers; from OpenAlex and the papers' affiliations).
- Xiamen University · 2 papers
Use it
Download BibTeX of 21 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Boundary-aware instance segmentation (83 papers)
- Joint detection and instance segmentation (39 papers)
Elsewhere in the topic
- Context modeling for semantic segmentation (241 papers)
- Region-based and query-based detectors (183 papers)
- Local features and context in CNNs (170 papers)
- Context for hard detection conditions (133 papers)
- Domain-specific segmentation and detection (118 papers)
- Convolution-transformer backbones (101 papers)
- Box regression and loss design (94 papers)
- Efficient DETR query design (85 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).