Research map / Detection and segmentation / Open-set and few-shot detection
DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM
Yixuan Wu, Yizhou Wang, Shixiang Tang, Wenhao Wu, Tong He, Wanli Ouyang, Philip Torr, Jian Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DetToolChain to GPT-4V on COCO open-vocabulary detection | GPT-4V · COCO OVD Novel | AP50: 2.3 -> 65.8 | Improved |
| add DetToolChain to GPT-4V on COCO val2017 | GPT-4V · COCO val2017 | AP: 0.3 -> 34.5 | Improved |
| add DetToolChain to GPT-4V on described object detection | GPT-4V · D-cube FULL | AP: 16.3 -> 37.2 | Improved |
| add DetToolChain to GPT-4V on RefCOCO | GPT-4V · RefCOCO val | Accuracy@0.5: 25.48 -> 70.01 | Improved |
| add ruler marker (ablation) | GPT-4V · COCO single object subset | mIoUp: 0.23 -> 0.52 | Improved |
| add compass marker for oriented objects (ablation) | GPT-4V · HRSC2016 | mIoUp: 0.13 -> 0.47 | Improved |
| add compass marker for COCO small objects (ablation) | GPT-4V · COCO small object subset | mIoUp: 0.16 -> 0.14 | No clear effect |
| add image split and zoom in (ablation) | GPT-4V · COCO small object subset | mIoUp: 0.16 -> 0.46 | Improved |
| add convex hull marker and self-verification to ruler (ablation) | GPT-4V · COCO occlusion subset | mIoUp: 0.43 -> 0.70 | Improved |
| use full DetToolChain instead of zero-shot CoT (ablation) | GPT-4V · COCO single object subset | mIoUp: 0.25 -> 0.78 | Improved |
| use DetToolChain instead of multimodal CoT (ablation) | GPT-4V · COCO multi-objects subset | mIoUp: 0.32 -> 0.64 | Improved |
| DetToolChain higher AP50 than AP75 on COCO val2017 | GPT-4V · COCO val2017 | AP75: AP75 31.5 vs DETR-R50 44.2 | Mixed |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- CORA — CORA: Adapting CLIP for Open-Vocabulary Detection With Region Prompting and Anchor Pre-Matching (CVPR 2023) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ROD-MLLM: Towards More Reliable Object Detection in Multimodal Large Language Models (CVPR 2025 · similarity 0.89)
- CP-DETR: Concept Prompt Guide DETR Toward Stronger Universal Object Detection (AAAI 2025 · similarity 0.88)
- Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection (CVPR 2024 · similarity 0.87)
- VP-Bench: A Comprehensive Benchmark for Visual Prompting in Multimodal Large Language Models (AAAI 2026 · similarity 0.86)
- EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detection (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
