Research map / Detection and segmentation / CNN features
Mr. DETR: Instructive Multi-Route Training for Detection Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Mr. DETR, a multi-route training scheme for detection transformers that adds two auxiliary one-to-many routes (one with an independent FFN, one with a learnable instructive self-attention) alongside the one-to-one primary route, discarding the auxiliary routes at inference.
Problem
DETR-style detectors trained with one-to-one assignment converge slowly due to sparse supervision, and adding one-to-many auxiliary training naively hurts one-to-one performance when decoder components are shared.
Approach
Treat the decoder with auxiliary one-to-many training as a multi-task framework; show any independent decoder component (self-attention, cross-attention, or FFN) lets one-to-one and one-to-many targets coexist; build a multi-route training scheme with an independent FFN route and a parameter-sharing instructive self-attention route that appends learnable instruction tokens to object queries, remov
Key results
- On COCO val with ResNet-50, Mr. DETR improves Deformable-DETR++ (300 queries, 12 epochs) from 47.0 to 49.5 mAP and (900 queries, 24 epochs) to 51.4 mAP (+1.6).
- Mr. DETR boosts Align-DETR to 52.3 mAP at 24 epochs (+1.0) and DINO to 50.9 mAP at 12 epochs (+1.9) with ResNet-50.
- With Swin-L backbone, Mr. DETR built on DINO reaches 58.4 mAP at 12 epochs, above the compared methods in Table 3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-route training | Deformable-DETR++ · COCO 2017 val, 12 ep, 900 queries | mAP: 47.6 -> 50.7 | Helped |
| add multi-route training | DINO · COCO 2017 val, 12 ep, 900 queries | mAP: 49.0 -> 50.9 | Helped |
| add multi-route training | Align-DETR · COCO 2017 val, 24 ep, 900 queries | mAP: 51.3 -> 52.3 | Helped |
| add multi-route training | Deformable-DETR++ instance segmentation · COCO 2017 val, 12 ep | Mask mAP: 32.4 -> 36.0 | Helped |
| share all decoder components across one-to-one and one-to-many tasks (ablation) | Deformable-DETR++ · COCO 2017 val, 12 ep, 900 queries | mAP (o2o): 47.6 -> 41.6 | Hurt |
| add an independent self-attention component (ablation) | Deformable-DETR++ multi-task decoder · COCO 2017 val, 12 ep, 900 queries | mAP (o2o): +2.1 | Helped |
| add an independent cross-attention component (ablation) | Deformable-DETR++ multi-task decoder · COCO 2017 val, 12 ep, 900 queries | mAP (o2o): +1.6 | Helped |
| add an independent FFN component (ablation) | Deformable-DETR++ multi-task decoder · COCO 2017 val, 12 ep, 900 queries | mAP (o2o): +2.0 | Helped |
| combine independent self-attention and FFN routes (ablation) | Deformable-DETR++ multi-task decoder · COCO 2017 val, 12 ep, 900 queries | mAP (o2o): +2.4 | Helped |
| add an auxiliary route with independent FFN (ablation) | Deformable-DETR++ · COCO 2017 val, 12 ep, 900 queries | mAP: +2.0 | Helped |
| add an auxiliary route with instructive self-attention (ablation) | Deformable-DETR++ · COCO 2017 val, 12 ep, 900 queries | mAP: +2.8 | Helped |
| replace concatenated instruction tokens with independent self-attention (ablation) | Deformable-DETR++ Route-3 · COCO 2017 val, 12 ep, 900 queries | mAP: 50.0 vs 50.7 | Hurt |
Surprising findings
- Sharing all decoder components between one-to-one and one-to-many targets drops one-to-one AP by 6.0 (47.6 to 41.6), while any single independent component recovers and improves it.
- Adding an auxiliary route with independent cross-attention degrades the primary route's performance, attributed to slow convergence of independent cross-attention.
Limitations
- Auxiliary routes add training-time cost and trainable parameters, which are not quantified.
- Gains are evaluated mainly on COCO 2017 with a fixed one-to-many assigner (K=6, alpha=0.3, tau=0.4) and hyperparameters tuned on this setup.
Experimental setup
Datasets: COCO 2017
Benchmarks: COCO 2017 val
Metrics: mAP AP50 AP75 APs APm APl mask mAP
Baselines: Deformable-DETR++ H-DETR MS-DETR DAC-DETR DINO Group-DETR Align-DETR Stable-DINO
Builds on: DETR Deformable DETR DINO DAC-DETR MS-DETR
Closest papers by meaning
- MS-DETR: Efficient DETR Training with Mixed Supervision (CVPR 2024 · similarity 0.87)
- Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement (CVPR 2024 · similarity 0.86)
- KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling (CVPR 2024 · similarity 0.86)
- How Transformers Learn Structured Data: Insights From Hierarchical Filtering (ICML 2025 · similarity 0.86)
- One-Layer Transformers Provably Learn In-Context K-Nearest Neighbor Prediction with Chain-of-Thought (NeurIPS 2026 · similarity 0.85)
- Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract) (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.