Research map / Continual learning / Class-incremental
SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDDGR, a diffusion-based generative replay method for class incremental object detection that uses GLIGEN-controlled Stable Diffusion with iterative refinement, L2 distillation, and pseudo-labeling on Deformable DETR.
Problem
Class incremental object detection (CIOD) suffers catastrophic forgetting, and generative replay has been hard to apply because multi-object scenes are hard to generate faithfully and real old-class data may be unavailable.
Approach
Generate multi-object synthetic images of old classes with a GLIGEN-grounded Stable Diffusion v1.4 (class labels, bounding boxes, text prompt, CLIP image embedding), filter them with the previous detector via iterative class-wise thresholding, and transfer knowledge via L2 distillation on decoder outputs of all layers, plus pseudo-labeling of old objects in new-task images.
Key results
- Two-phase 40+40 setting: AP 43.0 vs 42.0 for CL-DETR (+1.0), AP.5 62.1 vs 60.1 (+2.0).
- Two-phase 70+10 setting: AP 40.9 vs 40.4 for CL-DETR (+0.5), AP.5 59.5 vs 58.0.
- Multi-phase 40+10+10+10+10 final AP 36.8 vs 28.1 for CL-DETR (+8.7); 40+20+20 final AP 41.1 vs 35.3 (+5.8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SDDGR instead of CL-DETR | Deformable DETR · COCO 2017, 40+40 | AP: 42.0 -> 43.0 | Helped |
| Use SDDGR instead of CL-DETR | Deformable DETR · COCO 2017, 70+10 | AP: 40.4 -> 40.9 | Helped |
| Use SDDGR instead of CL-DETR | Deformable DETR · COCO 2017, 40+10+10+10+10, T5 | AP: 28.1 -> 36.8 | Helped |
| Use SDDGR instead of CL-DETR | Deformable DETR · COCO 2017, 40+20+20, T3 | AP: 35.3 -> 41.1 | Helped |
| Add pseudo-labeling (ablation) | Fine-tuning on COCO 2017 70+10 · COCO 2017, 70+10 | AP (all categories): 14.8 -> 38.6 | Helped |
| Add deep generative replay (ablation) | Pseudo-labeling method on COCO 2017 70+10 · COCO 2017, 70+10 | AP (all categories): 38.6 -> 39.8 | Helped |
| Add knowledge distillation (ablation) | Deep generative replay on COCO 2017 70+10 · COCO 2017, 70+10 | AP (all categories): 39.8 -> 40.9 | Helped |
| Set pseudo-label confidence threshold to 0.3 instead of 0.2 (ablation) | SDDGR on COCO 2017 70+10 · COCO 2017, 70+10 | AP: 29.5 -> 38.6 | Helped |
| Raise pseudo-label confidence threshold from 0.3 to 0.4 (ablation) | SDDGR on COCO 2017 70+10 · COCO 2017, 70+10 | AP: 38.6 -> 37.5 | Hurt |
| Remove the class-wise generated-image limit (ablation) | SDDGR on COCO 2017 70+10 · COCO 2017, 70+10 | AP: 40.9 -> 39.9 | Hurt |
| Use dynamic refinement threshold 0.4–0.8 rather than fixed 0.4 (ablation) | SDDGR on COCO 2017 70+10 · COCO 2017, 70+10 | AP: 39.8 -> 40.9 | Helped |
| Use dynamic refinement threshold 0.4–0.8 rather than fixed 0.8 (ablation) | SDDGR on COCO 2017 70+10 · COCO 2017, 70+10 | AP: 38.5 -> 40.9 | Helped |
Surprising findings
- Naively applying Stable Diffusion to CIOD decreased accuracy, motivating the refinement and distillation components.
- Without CLIP image embedding, synthetic-data training performed similarly to pseudo-labeling alone (38.6 AP).
Limitations
- Naive SD generation is insufficient for CIOD and requires filtering with the previous detector.
- Relies on GLIGEN grounding and a previous model for refinement and pseudo-labeling, adding pipeline complexity.
- Evaluated only on COCO 2017 with ResNet-50 Deformable DETR.
Experimental setup
Datasets: MS COCO 2017
Benchmarks: MS COCO 2017
Metrics: AP AP.5 AP.75 APS APM APL forgetting percentage points (FPP)
Baselines: LWF RILOD SID ERD CL-DETR
Builds on: GLIGEN Deformable DETR Stable Diffusion CL-DETR ERD
Closest papers by meaning
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.94)
- InstaGen: Enhancing Object Detection by Training on Synthetic Dataset (CVPR 2024 · similarity 0.87)
- Learning Task-Aware Language-Image Representation for Class-Incremental Object Detection (AAAI 2024 · similarity 0.86)
- Demystifying Catastrophic Forgetting in Two-Stage Incremental Object Detector (ICML 2025 · similarity 0.86)
- Tensor Decomposition Based Memory-Efficient Incremental Learning (ICML 2025 · similarity 0.85)
- ODGEN: Domain-specific Object Detection Data Generation with Diffusion Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.