Research map / Continual learning / Class-incremental
Adaptive Prototype Replay for Class Incremental Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adapter, which adaptively updates stored old-class prototypes for replay in class incremental semantic segmentation using a training-free deviation compensation plus uncertainty-aware and prototype discrimination losses.
Problem
Prototype replay methods for class incremental semantic segmentation keep prototypes fixed while the representation of old classes drifts as shared weights are updated, limiting replay effectiveness.
Approach
Estimate old-class representation shift between previous and current models on current-step images (training-free) and update stored prototypes accordingly (ADC); add an uncertainty-aware constraint loss to compact class representations and a compensation-based prototype similarity discrimination loss to separate new-class features from compensated old prototypes.
Key results
- On Pascal VOC 2012 overlapped 15-1 (6 steps), Adapter reaches 73.2 mIoU all-classes, versus 72.5 for STAR.
- On overlapped 2-2 (10 steps) and 1-1 (20 steps) VOC, Adapter surpasses prior state of the art by 3.0 and 6.2 mIoU, respectively.
- On ADE20K overlapped 100-5 (11 steps), Adapter reaches 34.5 mIoU, a 0.8 gain over the second-best method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Adapter instead of STAR | CISS segmentation · PASCAL VOC overlapped 2-2 (10 steps) | mIoU, all classes: 55.6 -> 58.6 | Helped |
| Use Adapter instead of STAR | CISS segmentation · PASCAL VOC overlapped 1-1 (20 steps) | mIoU, all classes: 36.5 -> 42.7 | Helped |
| Use Adapter instead of STAR | CISS segmentation · ADE20K overlapped 100-5 (11 steps) | mIoU, all classes: 33.7 -> 34.5 | Helped |
| Add ADC to fixed prototype replay baseline (ablation) | CISS segmentation · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 71.3 -> 71.9 | Helped |
| Add UAC after ADC (ablation) | CISS segmentation · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 71.9 -> 72.4 | Helped |
| Add CPD after ADC and UAC (ablation) | CISS segmentation · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 72.4 -> 73.2 | Helped |
| Add UAC without ADC or CPD (ablation) | CISS segmentation baseline · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 71.3 -> 71.8 | Helped |
| Add CPD without ADC or UAC (ablation) | CISS segmentation baseline · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 71.3 -> 72.4 | Helped |
| Add UAC and CPD without ADC (ablation) | CISS segmentation baseline · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 71.3 -> 72.7 | Helped |
| Add ADC and CPD without UAC (ablation) | CISS segmentation baseline · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 71.3 -> 72.9 | Helped |
| Use ADC-compensated rather than fixed prototypes in CPD (ablation) | Adapter CPD · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 72.9 -> 73.2 | Helped |
| Add L(n↔o) to the no-CPD configuration (ablation) | Adapter CPD · PASCAL VOC overlapped 15-1 (6 steps) | mIoU, all classes: 72.4 -> 72.7 | Helped |
Limitations
- Relies on estimating representation shift from limited old-class pixels in current-step data, since old data is unavailable.
- Prediction-mask uncertainty can make compensated prototypes less reliable, which the method mitigates only partially.
- Hyperparameter sensitivity analysis is limited to 15-1 setting on VOC.
Experimental setup
Datasets: Pascal VOC 2012 ADE20K
Benchmarks: Pascal VOC 2012 ADE20K
Metrics: mIoU
Baselines: MiB PLOP SSUL DKD RCIL MicroSeg IDEC CoinSeg STAR UCD REMIND ALIFE
Builds on: MiB STAR PLOP SSUL
Closest papers by meaning
- Saving 100x Storage: Prototype Replay for Reconstructing Training Sample Distribution in Class-Incremental Semantic Segmentation (NeurIPS 2023 · similarity 0.93)
- Beyond Background Shift: Rethinking Instance Replay in Continual Semantic Segmentation (CVPR 2025 · similarity 0.89)
- Deconfound Semantic Shift and Incompleteness in Incremental Few-shot Semantic Segmentation (AAAI 2025 · similarity 0.87)
- CoMBO: Conflict Mitigation via Branched Optimization for Class Incremental Segmentation (CVPR 2025 · similarity 0.87)
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.86)
- Rethinking Query-based Transformer for Continual Image Segmentation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.