Research map / Detection and segmentation / CNN features
DEIM: DETR with Improved Matching for Fast Convergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DEIM, a DETR training framework combining Dense O2O matching (mosaic/mixup to add targets per image) with a Matchability-Aware Loss to speed convergence and boost real-time detection on COCO.
Problem
DETR-based detectors converge slowly because one-to-one matching gives sparse supervision and many low-quality matches from randomly or weakly initialized queries.
Approach
Increase the number of targets per training image via standard mosaic and mixup augmentation so one-to-one matching yields more positives, and replace Varifocal Loss with a Matchability-Aware Loss whose target label q^gamma scales penalties for low-IoU matches.
Key results
- DEIM-D-FINE-X reaches 56.5% AP on COCO val2017 at 78 FPS on a T4 GPU, and DEIM-D-FINE-L reaches 54.7% AP at 8.07 ms latency, exceeding YOLOv11-X (54.1 AP, 10.74 ms).
- DEIM with RT-DETRv2-R50 reaches 53.2% AP in about one day on a single RTX 4090, and 54.3% AP at 60 epochs versus 53.4% for the 72-epoch baseline.
- On CrowdHuman, D-FINE-L with DEIM improves AP from 56.0 to 57.5 at 120 epochs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train D-FINE-X with DEIM | D-FINE-X · COCO val2017 | AP: 55.8 -> 56.5 | Helped |
| Train RT-DETRv2-R50 with DEIM for 36 epochs | RT-DETRv2-R50 · COCO val2017, ResNet50 | AP: 53.4 -> 53.9 | Helped |
| Train RT-DETRv2-R101 with DEIM for 36 epochs | RT-DETRv2-R101 · COCO val2017, ResNet101 | AP: 54.3 -> 55.2 | Helped |
| Train D-FINE-L with DEIM | D-FINE-L · CrowdHuman | AP: 56.0 -> 57.5 | Helped |
| Add mosaic augmentation (ablation) | RT-DETRv2-R50 · COCO val2017, 12 epochs | AP: 49.6 -> 50.4 | Helped |
| Add mixup augmentation (ablation) | RT-DETRv2-R50 · COCO val2017, 12 epochs | AP: 49.6 -> 50.1 | Helped |
| Add mosaic and mixup augmentation (ablation) | RT-DETRv2-R50 · COCO val2017, 24 epochs | AP: 51.7 -> 52.5 | Helped |
| Set MAL γ to 1.5 rather than 1.3, 1.8, or 2.0 (ablation) | RT-DETRv2-R50 with MAL · COCO val2017, 24 epochs | AP: 52.2; 52.4; 52.1; 51.9 | Helped |
| Add Dense O2O (ablation) | RT-DETRv2-R50 · COCO val2017, 36 epochs | AP: 53.4 -> 53.6 | Helped |
| Add MAL after Dense O2O (ablation) | RT-DETRv2-R50 · COCO val2017, 36 epochs | AP: 53.6 -> 53.9 | Helped |
| Add Dense O2O (ablation) | D-FINE-L · COCO val2017, 36 epochs | AP: 54.0 -> 54.2 | Helped |
| Add MAL after Dense O2O (ablation) | D-FINE-L · COCO val2017, 36 epochs | AP: 54.2 -> 54.6 | Helped |
Surprising findings
- Dense O2O via mosaic and mixup adds positives at almost no extra training cost (1.183 vs 1.181 GPU hours per epoch), and reaches baseline accuracy in about half the epochs.
Limitations
- Small-object AP still trails some YOLO models (e.g., YOLOv9-E), indicating remaining challenges for DETR on small objects.
- Dense O2O relies on augmentation that produces many low-quality matches, which requires MAL to mitigate.
Experimental setup
Datasets: MS-COCO CrowdHuman ImageNet-1k Objects365
Benchmarks: COCO val2017 CrowdHuman
Metrics: AP AP50 AP75 APs APm APl latency (ms) GFLOPs GPU hours
Baselines: RT-DETRv2 D-FINE YOLOv11 YOLOv10 YOLOv9 YOLOv8 DINO-Deformable-DETR Group DETR Co-DETR
Builds on: DETR RT-DETRv2 D-FINE Varifocal Loss Mosaic augmentation Mixup
Closest papers by meaning
- DETRs Beat YOLOs on Real-time Object Detection (CVPR 2024 · similarity 0.87)
- MS-DETR: Efficient DETR Training with Mixed Supervision (CVPR 2024 · similarity 0.85)
- YOLOv10: Real-Time End-to-End Object Detection (NeurIPS 2024 · similarity 0.85)
- MI-DETR: An Object Detection Model with Multi-time Inquiries Mechanism (CVPR 2025 · similarity 0.85)
- KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling (CVPR 2024 · similarity 0.84)
- Open-Det: An Efficient Learning Framework for Open-Ended Detection (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.