Research map / Detection and segmentation / CNN features
MS-DETR: Efficient DETR Training with Mixed Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MS-DETR, which adds one-to-many supervision directly on the primary decoder's object queries alongside one-to-one supervision, improving DETR training without extra decoders or queries.
Problem
DETR training with one-to-one supervision lacks direct supervision for generating object detection candidates, making training slow and candidate quality limited.
Approach
Mixes one-to-one and one-to-many supervision on the object queries of the primary (inference) decoder via an extra box and class predictor pair used only in training, with one-to-many matching by a combined IoU and classification score and top-K selection with threshold; no additional decoder branches or queries are needed.
Key results
- On Deformable DETR++ (12 epochs), MS-DETR reaches 48.8 mAP vs 47.0 baseline (+1.8); on DAB-Deformable-DETR it gains +3.7 mAP (44.2 to 47.9).
- Combined with DINO, MS-DETR improves 12-epoch mAP from 49.0 to 50.3 and 24-epoch mAP from 50.4 to 51.7.
- Training overhead is minimal: 69 min per epoch vs 67 min baseline and 5243M vs 5116M GPU memory, versus Hybrid DETR at 103 min and 8680M.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add one-to-many supervision to the primary decoder | DAB-Deformable-DETR · COCO val2017, 12 epochs | mAP: 44.2 -> 47.9 (+3.7) | Helped |
| Add one-to-many supervision to the primary decoder | Deformable DETR · COCO val2017, 12 epochs | mAP: 43.7 -> 47.6 (+3.7) | Helped |
| Add one-to-many supervision to the primary decoder | Deformable DETR++ · COCO val2017, 12 epochs | mAP: 47.0 -> 48.8 (+1.8) | Helped |
| Add MS-DETR mixed supervision | DINO · COCO val2017, 12 epochs | mAP: 49.0 -> 50.3 (+1.3) | Helped |
| Add MS-DETR mixed supervision | Align-DETR · COCO val2017, 12 epochs | mAP: 50.2 -> 50.7 (+0.5) | Helped |
| Apply one-to-many supervision to internal queries after cross-attention rather than output queries (ablation) | Deformable DETR with MS-DETR · COCO val2017 | mAP: 47.0 -> 47.6 | Helped |
| Apply supervision to internal queries after self-attention rather than after cross-attention (ablation) | Deformable DETR with MS-DETR · COCO val2017 | mAP: 47.6 -> 46.1 | Hurt |
| Share both box and class predictor weights rather than neither (ablation) | Deformable DETR with MS-DETR · COCO val2017 | mAP: 47.2 -> 47.6 | Helped |
| Vary the top-K positive-query selection; the paper reports the best result at K = 6 (ablation) | MS-DETR · COCO val2017 | mAP: 47.2, 47.4, 47.6, 47.3, 47.2 | Mixed |
| Vary the matching threshold τ; the paper reports the best result at τ = 0.4 (ablation) | MS-DETR · COCO val2017 | mAP: 47.3, 47.5, 47.6, 47.2, 46.9 | Mixed |
| Vary matching-score weight α; the paper reports the best result at α = 0.4 (ablation) | MS-DETR · COCO val2017 | mAP: 47.1, 47.4, 47.6, 47.3, 47 | Mixed |
| Add MS-DETR mixed supervision | Mask-Deformable-DETR · COCO-2017 val, 12 epochs | Mask mAP: 28.3 -> 31.5 (+3.2) | Helped |
Surprising findings
- Sharing both box and class predictors between one-to-many and one-to-one supervision gives the best performance (47.6 mAP), while class-predictor-only sharing performs worse than sharing both.
Limitations
- Gains on stronger baselines are smaller (e.g., +1.8 mAP on Deformable DETR++ at 12 epochs)
- Evaluation is limited to COCO and ResNet-50 backbone in the main experiments
Experimental setup
Datasets: COCO train2017
Benchmarks: COCO val2017
Metrics: mAP AP50 AP75 APs APm APl box mAP mask mAP training time GPU memory
Baselines: DAB-Deformable-DETR Deformable DETR Deformable DETR++ DN-DETR Group DETR Hybrid DETR DINO DAC-DETR Align-DETR DETA
Builds on: DETR Deformable DETR DAB-DETR DN-DETR Hybrid DETR
Closest papers by meaning
- MI-DETR: An Object Detection Model with Multi-time Inquiries Mechanism (CVPR 2025 · similarity 0.89)
- Order Matters: Competition-Guided Query Ordering for RNN-Based Object Detection (NeurIPS 2026 · similarity 0.87)
- CP-DETR: Concept Prompt Guide DETR Toward Stronger Universal Object Detection (AAAI 2025 · similarity 0.87)
- Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection (CVPR 2024 · similarity 0.87)
- UN-DETR: Promoting Objectness Learning via Joint Supervision for Unknown Object Detection (AAAI 2025 · similarity 0.87)
- Hybrid Proposal Refiner: Revisiting DETR Series from the Faster R-CNN Perspective (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.