Research map / Continual learning / Class-incremental
Learning Task-Aware Language-Image Representation for Class-Incremental Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies a GLIP language-image detector to class-incremental object detection and learns task-specific channel/hidden-state masks (TAM) with a row-max selective inference strategy (SIS) to reduce background-foreground conflict and catastrophic forgetting.
Problem
Class-incremental object detection suffers from catastrophic forgetting, aggravated by background-foreground conflict in visual-only detectors; language-image detectors help but still forget in challenging many-class incremental settings.
Approach
Partition image feature channels and text hidden states into disjoint task-specific subsets via a Task-Aware Module during training, so each task learns an independent-like representation; at inference, use only the task-specific portion of alignment scores and combine them with a row-wise max (Selective Inference Strategy).
Key results
- On COCO 2017 70+10 the method reaches 42.9 AP vs 37.6/40.1 for CL-DETR, and 40.4 AP on 40+40 vs 37.0/37.5 for CL-DETR.
- On VOC 2007 10+10 setting the method achieves 70.6 average mAP@0.5, outperforming CIOD (67.1) and other visual-only methods.
- Ablation on VOC 10+10 shows the selective inference strategy (row-max) yields 70.6 vs 50.9 (max) and 52.3 (average) when used on the full-branch TAM model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply task-aware representation and selective inference instead of fine-tuning alone (ablation) | GLIP on Pascal VOC 2007 10+10 · Pascal VOC 2007 10+10 | Avg mAP@0.5: 50.6 -> 70.6 | Helped |
| apply TAM to the visual branch only (ablation) | GLIP on Pascal VOC 2007 10+10 · Pascal VOC 2007 10+10 | Avg mAP@0.5: 50.6 -> 53.1 | Helped |
| apply TAM to the language branch only (ablation) | GLIP on Pascal VOC 2007 10+10 · Pascal VOC 2007 10+10 | Avg mAP@0.5: 50.6 -> 56.6 | Helped |
| use TAM on both branches with selective inference instead of TAM on the language branch only (ablation) | GLIP on Pascal VOC 2007 10+10 · Pascal VOC 2007 10+10 | Avg mAP@0.5: 56.6 -> 70.6 | Helped |
| replace selective inference with element-wise mean inference (ablation) | TAM on both GLIP branches · Pascal VOC 2007 10+10 | Avg mAP@0.5: 70.6 -> 52.3 | Hurt |
| replace selective inference with element-wise max inference (ablation) | TAM on both GLIP branches · Pascal VOC 2007 10+10 | Avg mAP@0.5: 70.6 -> 50.9 | Hurt |
| use independent task models with selective inference rather than TAM (ablation) | GLIP on Pascal VOC 2007 10+10 · Pascal VOC 2007 10+10 | Avg mAP@0.5: 70.6 -> 72.2 | Helped |
| allocate first 75% of channels and hidden states to task 1 instead of a 50% split (ablation) | TAM on GLIP · Pascal VOC 2007 10+10 | Avg mAP@0.5: 70.6 -> 70.0 | Hurt |
| allocate first 25% of channels and hidden states to task 1 instead of a 50% split (ablation) | TAM on GLIP · Pascal VOC 2007 10+10 | Avg mAP@0.5: 70.6 -> 69.9 | Hurt |
| randomly select 50% of channels and hidden states rather than using consecutive halves (ablation) | TAM on GLIP · Pascal VOC 2007 10+10 | Avg mAP@0.5: 70.6 vs 70.4 | No clear effect |
| use task-aware language-image representation and selective inference | GLIP on COCO 2017 70+10 · COCO 2017 70+10 | AP: 40.1 -> 42.9 | Helped |
| use task-aware language-image representation and selective inference | GLIP on COCO 2017 40+40 · COCO 2017 40+40 | AP: 37.5 -> 40.4 | Helped |
Surprising findings
- Naive GLIP as a language-image CIOD baseline already greatly outperforms visual-only detectors in fewer-category settings, yet still forgets in challenging settings such as 40+40 on COCO.
Limitations
- Performance on many-class settings still shows forgetting relative to upper bound in some settings.
- Channel/hidden-state split is proportional to class counts and manually chosen per setting.
Experimental setup
Datasets: COCO 2017 Pascal VOC 2007
Benchmarks: COCO 2017 Pascal VOC 2007
Metrics: AP AP50 AP75 APS APM APL mAP@0.5
Baselines: SID ERD CL-DETR LOD Meta MVCD CIOD Fine-tuning
Builds on: GLIP Der CL-DETR ERD
Closest papers by meaning
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.89)
- Class Incremental Learning via Likelihood Ratio Based Task Prediction (ICLR 2024 · similarity 0.89)
- DCA: Dividing and Conquering Amnesia in Incremental Object Detection (AAAI 2025 · similarity 0.88)
- Demystifying Catastrophic Forgetting in Two-Stage Incremental Object Detector (ICML 2025 · similarity 0.87)
- Enhancing Visual Continual Learning with Language-Guided Supervision (CVPR 2024 · similarity 0.87)
- SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.