Research map / Detection and segmentation / CNN features
MatchDet: A Collaborative Framework for Image Matching and Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MatchDet, a task-collaborative framework that jointly trains image matching (LoFTR-style) and object detection (FCOS-style) on a shared ResNet-50 backbone, using weighted attention, weighted spatial attention, and a Box Filter so the two tasks improve each other, evaluated on Warp-COCO and a new miniScanNet benchmark.
Problem
Image matching and object detection are usually solved as separate tasks, although applications such as SLAM and defect inspection need both; the paper asks whether jointly learning them can mutually improve each.
Approach
A shared backbone with a Matcher and Detector co-trained end-to-end; Weighted Attention (Matcher) and Weighted Spatial Attention (Detector) use foreground weight maps derived from boxes or segmentation to focus attention on object regions, and a Box Filter uses detected boxes to suppress false matches.</:parameter>
Key results
- On Full Warp-COCO (GTBoxR setting), MatchDet-G reaches 43.62 AP vs 30.95 for MDBase and 63.18 AUC@3px vs 36.42 for MDBase.
- On miniScanNet, MatchDet-G reaches 40.69 AP and 18.43 AUC@3px, compared with 23.70 AP and 5.95 AUC@3px for MDBase.
- With FCOS, Faster-RCNN and AdaMixer detectors, MatchDet-G improves AP over each detector alone (e.g., AdaMixer 40.82 to 48.19 on COCO/Warp-COCO).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MatchDet collaborative modules in the GTBoxR setting | MDBase · Warp-COCO, COCO | AP: 30.95 -> 43.62 | Helped |
| add MatchDet collaborative modules in the GTBoxR setting | MDBase · Warp-COCO, COCO | Homography AUC@3px: 36.42 -> 63.18 | Helped |
| add MatchDet collaborative modules in the GTBoxR setting | MDBase · miniScanNet-F0, miniScanNet-F1 | AP: 23.70 -> 40.69 | Helped |
| add MatchDet collaborative modules in the GTBoxR setting | MDBase · miniScanNet-F0, miniScanNet-F1 | Homography AUC@3px: 5.95 -> 18.43 | Helped |
| add WSAM (ablation) | MatchDet-G with WAM · Warp-COCO, COCO | AP: 30.66 -> 43.62 | Helped |
| add WAM (ablation) | MatchDet-G with WSAM · Warp-COCO, COCO | Homography AUC@3px: 60.12 -> 61.15 | Helped |
| add Box Filter (ablation) | MatchDet-G with WAM and WSAM · Warp-COCO, COCO | Homography AUC@3px: 61.15 -> 63.18 | Helped |
| replace Cross Attention with Weighted Attention and Weighted Spatial Attention (ablation) | MatchDet · Warp-COCO, COCO | AP: 31.01 -> 43.62 | Helped |
| replace Cross Attention with Weighted Attention and Weighted Spatial Attention (ablation) | MatchDet · Warp-COCO, COCO | Homography AUC@3px: 45.84 -> 61.15 | Helped |
| use Weighted Spatial Attention in WAM instead of Weighted Attention (ablation) | MatchDet · Warp-COCO, COCO | Homography AUC@3px: 61.15 -> 58.46 | Hurt |
| use Weighted Attention in WSAM instead of Weighted Spatial Attention (ablation) | MatchDet · Warp-COCO, COCO | AP: 43.62 -> 40.41 | Hurt |
| add a second WAM while retaining one WSAM (ablation) | MatchDet-G · Warp-COCO, COCO | Homography AUC@3px: 57.93 -> 61.15 | Helped |
Surprising findings
- Adding only the WAM to the matcher without WSAM lowers AP slightly (30.66 vs 30.95 for MDBase) while greatly raising matching AUC, indicating the trade-off between tasks is not uniform.
Limitations
- miniScanNet is a new benchmark that is not released per the paper text
- Under the NoBoxR setting gains are smaller and accuracy is lower than the GTBoxR setting
Experimental setup
Datasets: COCO Warp-COCO miniScanNet ScanNet
Benchmarks: Warp-COCO miniScanNet
Metrics: AP AP50 AP75 AUC of corner error@3px/5px/10px
Baselines: FCOS LoFTR Faster-RCNN AdaMixer Tracker MDBase
Builds on: LoFTR FCOS BoxInst Mask2Former
Closest papers by meaning
- PairDETR : Joint Detection and Association of Human Bodies and Faces (CVPR 2024 · similarity 0.86)
- Search and Detect: Training-Free Long Tail Object Detection via Web-Image Retrieval (CVPR 2025 · similarity 0.86)
- Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task (CVPR 2020 · similarity 0.85)
- InstructDET: Diversifying Referring Object Detection with Generalized Instructions (ICLR 2024 · similarity 0.85)
- DetKDS: Knowledge Distillation Search for Object Detectors (ICML 2024 · similarity 0.84)
- OmniGlue: Generalizable Feature Matching with Foundation Model Guidance (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.