Research map / Continual learning / Class-incremental
Demystifying Catastrophic Forgetting in Two-Stage Incremental Object Detector
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes Faster R-CNN forgetting in incremental object detection, finds it concentrated in the RoI Head classifier, and proposes NSGP-RePRE (regional prototype replay plus null-space gradient projection) reaching SOTA on Pascal VOC and MS COCO.
Problem
Catastrophic forgetting in two-stage incremental object detection (IOD) is typically handled for the detector as a whole, without analyzing which Faster R-CNN component forgets.
Approach
Component-wise dissection shows forgetting is mainly in the RoI Head classifier while regressors and RPN retain knowledge; RePRE replays coarse class-mean and density-selected fine-grained RoI prototypes for the classifier, and NSGP projects backbone/neck gradients into the null space of old inputs so stored prototypes stay aligned with RoI features.
Key results
- On VOC 19-1, 15-5, 10-10, 5-15 single-step settings, NSGP-RePRE reports Avg 72.7, 69.7, 74.0, 71.5, exceeding BPF's Avg on most settings.
- On COCO 40-40, NSGP-RePRE reaches AP 35.4 vs BPF 34.4 and ABR 34.5; on 70-10 it reaches AP 36.5 vs BPF 36.2.
- Ablation on VOC 5-5: adding NSGP to the baseline lifts 1-20 mAP from 58.4 to 63.3; full model with coarse and fine prototypes reaches 65.7.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace BPF with NSGP-RePRE | Faster R-CNN incremental detection · PASCAL VOC 2007 5-5 (4 tasks), 1-20 | mAP@0.5: 62.5 -> 65.7 | Helped |
| replace ABR with NSGP-RePRE | Faster R-CNN incremental detection · PASCAL VOC 2007 10-1 (11 tasks), 1-20 | mAP@0.5: 58.9 -> 62.5 | Helped |
| replace ABR with NSGP-RePRE | Faster R-CNN incremental detection · MS COCO 2017 40-40 | AP: 34.5 -> 35.4 | Helped |
| replace BPF with NSGP-RePRE | Faster R-CNN incremental detection · MS COCO 2017 70-10 | AP: 36.2 -> 36.5 | Helped |
| add NSGP without prototype replay (ablation) | pseudo-labeling baseline · PASCAL VOC 2007 5-5, 1-20 | mAP@0.5: 58.4 -> 63.3 | Helped |
| add coarse prototype replay without NSGP (ablation) | pseudo-labeling baseline · PASCAL VOC 2007 5-5, 1-20 | mAP@0.5: 58.4 -> 61.2 | Helped |
| add NSGP to coarse prototype replay (ablation) | coarse-prototype RePRE · PASCAL VOC 2007 5-5, 1-20 | mAP@0.5: 61.2 -> 64.3 | Helped |
| add fine-grained prototypes to NSGP and coarse replay (ablation) | NSGP + coarse prototype replay · PASCAL VOC 2007 5-5, 1-20 | mAP@0.5: 64.3 -> 65.7 | Helped |
| generate fine-grained prototypes with the proposed density-aware averaging strategy instead of K-Means (ablation) | NSGP-RePRE · PASCAL VOC 2007 5-5, 1-20 | mAP@0.5: 64.9 -> 65.7 | Helped |
| average features in hyperspheres instead of using hypersphere centers as instances (ablation) | NSGP-RePRE · PASCAL VOC 2007 5-5, 1-20 | mAP@0.5: 64.3 -> 65.7 | Helped |
| use normalized projection for neck, RPN, or RoI Head instead of the unnormalized projection (ablation) | NSGP · PASCAL VOC 2007 5-5 | mAP@0.5: 63.3 -> 60.3; 63.0 -> 59.0; 63.2 -> 41.9 | Hurt |
| restrict NSGP to backbone and neck rather than applying it to additional detector components (ablation) | NSGP · PASCAL VOC 2007 5-5 | mAP@0.5: 63.3; 63.0; 63.2 | Mixed |
Surprising findings
- RPN and regressor forgetting is minimal, and the reason for minimal regression forgetting versus severe classifier forgetting is stated as unclear by the authors.
- RPN recall on unseen classes improves after sequential training, suggesting zero-shot generalization.
Limitations
- Forgetting is analyzed only on VOC with a single backbone (ResNet-50) and Faster R-CNN; generality to other detectors is not shown.
- The authors state the cause of minimal regression forgetting is unclear.
- NSGP projection matrix is adapted with ad hoc choices (unnormalized B for non-backbone components), with sensitivity shown only on VOC 5-5.
Experimental setup
Datasets: PASCAL VOC 2007 MS COCO 2017
Benchmarks: PASCAL VOC 2007 MS COCO 2017
Metrics: mAP@0.5 AP AP50 AP75
Baselines: Fine-tuning Joint ORE OW-DETR ILOD-Meta ABR FasterILOD PPAS MVC PROB PseudoRM MMA
Builds on: Faster R-CNN BPF ABR Gradient Projection Memory / null space projection EWC
Closest papers by meaning
- DCA: Dividing and Conquering Amnesia in Incremental Object Detection (AAAI 2025 · similarity 0.91)
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.90)
- Learning Task-Aware Language-Image Representation for Class-Incremental Object Detection (AAAI 2024 · similarity 0.87)
- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks (ICML 2025 · similarity 0.86)
- SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection (CVPR 2024 · similarity 0.86)
- Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.