Research map / Detection and segmentation / CNN features
Learning Endogenous Attention for Incremental Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LEA, an annotation-to-instance alignment paradigm for incremental object detection using lightweight endogenous attention modules per task and an energy-based task-ID predictor on a frozen ViTDet backbone.
Problem
Incremental object detection suffers from the annotation incomplete problem: old-class objects are unlabeled (treated as background) in new tasks, causing semantic conflicts and catastrophic forgetting.
Approach
Instead of completing missing old-class annotations (image-to-annotation alignment), the model focuses on each annotated instance via per-task endogenous attention modules added to a frozen ViTDet, and an energy-based task modulator predicts task IDs at test time to select which modules detect objects.
Key results
- On COCO 70+10 two-task setting, LEA reaches 45.5 AP vs. 35.7 for ABR and 42.9 for TLR.
- On VOC 2007 15+5, LEA reaches 73.5 AP50, beating DTD's 70.6.
- ETM achieves F1 above 84 in all two-task COCO/VOC settings, while oracle task IDs give modest AP50 gains (e.g., 65.3 vs 67.1 on COCO 70+10).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LEA instead of TLR | COCO 2017 70 + 10 · COCO 2017 70 + 10 | AP: 42.9 -> 45.5 | Helped |
| use LEA instead of TLR | COCO 2017 40 + 40 · COCO 2017 40 + 40 | AP: 40.4 -> 41.2 | Helped |
| use LEA instead of ABR | COCO 2017 40 + 4 × 10 · COCO 2017 40 + 4 × 10 | AP: 31.2 -> 39.3 | Helped |
| use LEA instead of ABR | COCO 2017 40 + 2 × 20 · COCO 2017 40 + 2 × 20 | AP: 32.7 -> 39.9 | Helped |
| use LEA instead of DTD | VOC 2007 15 + 5 · VOC 2007 15 + 5 | AP50: 70.6 -> 73.5 | Helped |
| use LEA instead of DTD | VOC 2007 10 + 10 · VOC 2007 10 + 10 | AP50: 70.9 -> 72.2 | Helped |
| add EAMs (ablation) | ViTDet, VOC 2007 10 + 10 · VOC 2007 10 + 10 | AP: 33.9 -> 48.8 | Helped |
| use EAMs in 3 rather than 6 transformer blocks (ablation) | ViTDet, VOC 2007 10 + 10 · VOC 2007 10 + 10 | AP: 48.8 -> 48.1 | Hurt |
| replace EAMs with AdaptFormer (ablation) | ViTDet, VOC 2007 10 + 10 · VOC 2007 10 + 10 | AP50: 72.2 -> 70.3 | Hurt |
| replace EAMs with LoRA (ablation) | ViTDet, VOC 2007 10 + 10 · VOC 2007 10 + 10 | AP50: 72.2 -> 61.2 | Hurt |
| replace ETM with logits-based task modulator (LTM) (ablation) | ViTDet, VOC 2007 10 + 10 · VOC 2007 10 + 10 | AP50: 72.2 -> 70.9 | Hurt |
| add task prompt to ETM (ablation) | ViTDet, VOC 2007 10 + 10 · VOC 2007 10 + 10 | AP50: 71.2 -> 72.2 | Helped |
Surprising findings
- Comparing against parameter-efficient methods, EAMs outperform AdaptFormer by only 1.9 AP50 while BitFit, SSF and LoRA trail by 11-13 AP50 points.
- Removing all EAMs (ND=0) drops AP to 33.9, showing the linear predictors alone cannot handle incremental tasks.
Limitations
- Task-ID prediction accuracy (ETM) declines as the number of incremental tasks grows, reducing detection AP relative to oracle task IDs.
- Relies on a ViT-B/16 backbone for ETM rather than a stronger vision-language model, which the authors identify as future improvement.
Experimental setup
Datasets: MS COCO 2017 Pascal VOC 2007 ImageNet
Benchmarks: MS COCO 2017 Pascal VOC 2007
Metrics: AP AP50 AP75 APS APM APL precision recall F1
Baselines: Fine-tuning ERD IOR CL-DETR TLR Faster ILOD MMA ABR DTD CIOD ILOD BitFit
Builds on: ViTDet AdaptFormer Mask R-CNN energy-based models
Closest papers by meaning
- Boosting Vision-Language Models Towards Cross-Domain Incremental Object Detection (CVPR 2026 · similarity 0.84)
- Incremental-DETR: Incremental Few-Shot Object Detection via Self-Supervised Learning (AAAI 2023 · similarity 0.84)
- YOLO-IOD: Towards Real Time Incremental Object Detection (AAAI 2026 · similarity 0.84)
- Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection (CVPR 2026 · similarity 0.83)
- Training-Free Open-Ended Object Detection and Segmentation via Attention as Prompts (NeurIPS 2024 · similarity 0.83)
- Adaptive Important Region Selection with Reinforced Hierarchical Search for Dense Object Detection (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.