Research map / Detection and segmentation / CNN features
Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Salience DETR, which filters and encodes only discriminative queries using scale-independent salience supervision, hierarchical layer- and level-wise filtering, and query refinement modules for two-stage DETR detectors.
Problem
Two-stage DETR-like detectors incur heavy computation from dense encoder self-attention and suffer scale bias and redundancy in two-stage query selection, especially for small weak objects in task-specific detection.
Approach
Replaces discrete foreground/background supervision with a scale-independent salience target (decaying from object center to border) to guide hierarchical layer-wise and level-wise query filtering, and adds background embedding, cross-level token fusion, and NMS-based redundancy removal to compensate for semantic misalignment among queries.
Key results
- Achieves 49.2% AP on COCO val2017 with 201G FLOPs versus DINO's 49.0% AP at 291G FLOPs (12 epochs, ResNet50).
- Improves AP by +4.4% on MSSD (55.4 vs 51.0 for DINO) and AP75 by 9.4% over the second-best method.
- Reaches 46.5 AP on ESD, outperforming the second-best method and becoming the only method with AP75 above 40%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Salience DETR instead of HTC | ResNet50 detector · ESD test | AP: 45.4 -> 46.5 | Helped |
| Use Salience DETR instead of DINO | ResNet50 detector · MSSD test | AP: 51.0 -> 55.4 | Helped |
| Use Salience DETR instead of DINO | ResNet50 detector · COCO val2017 | AP: 49.0 -> 49.2 | Helped |
| Use Salience DETR instead of DINO | ResNet50 detector · COCO val2017 | FLOPs: 291G -> 201G | Helped |
| Add hierarchical query filtering with scale-independent salience supervision (ablation) | Salience DETR · CSD test | AP: 50.2 -> 52.0 | Helped |
| Add background embedding to hierarchical query filtering (ablation) | Salience DETR · CSD test | AP: 52.0 -> 52.0 | No clear effect |
| Add redundancy removal to hierarchical query filtering (ablation) | Salience DETR · CSD test | AP: 52.0 -> 52.6 | Helped |
| Add background embedding and redundancy removal (ablation) | Salience DETR · CSD test | AP: 52.0 -> 52.8 | Helped |
| Add cross-level token fusion (ablation) | Salience DETR · CSD test | AP: 52.8 -> 53.2 | Helped |
| Use scale-independent rather than overlap-limit supervision (ablation) | Salience DETR · MSSD | AP: 53.2 -> 55.0 | Helped |
| Use absolute rather than relative background embedding (ablation) | Salience DETR · MSSD | AP: 55.0 -> 55.2 | Helped |
| Add redundancy removal (ablation) | Salience DETR · CSD | AP: +0.7% AP | Helped |
Surprising findings
- Scale-based query assignment (overlap limit) was outperformed by scale-independent salience supervision, which suggests that scale-tied filtering favors large objects.
Limitations
- Redundancy removal relies on non-maximum suppression, which the authors say should be replaced by end-to-end solutions.
- Adds parameters and memory relative to baseline (56.1M parameters, 249MB memory with cross-level token fusion versus 46M for DINO).
Experimental setup
Datasets: CSD ESD MSSD COCO 2017
Benchmarks: ESD CSD MSSD COCO 2017
Metrics: AP AP50 AP75 APS APM APL AR FLOPs
Baselines: DINO Focus DETR H-Def-DETR DN-Def-DETR Deformable DETR Faster RCNN Cascade RCNN HTC YOLOX Sparse DETR
Builds on: DINO Deformable DETR Focus DETR Sparse DETR DETR
Closest papers by meaning
- AQ-DETR: Low-Bit Quantized Detection Transformer with Auxiliary Queries (AAAI 2024 · similarity 0.88)
- EASE-DETR: Easing the Competition among Object Queries (CVPR 2024 · similarity 0.87)
- Mr. DETR: Instructive Multi-Route Training for Detection Transformers (CVPR 2025 · similarity 0.86)
- DetKDS: Knowledge Distillation Search for Object Detectors (ICML 2024 · similarity 0.86)
- KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling (CVPR 2024 · similarity 0.86)
- Order Matters: Competition-Guided Query Ordering for RNN-Based Object Detection (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.