Research map / Detection and segmentation / Open-set and few-shot detection
DiPEx: Dispersing Prompt Expansion for Class-Agnostic Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiPEx, a self-supervised prompt expansion method that grows a set of non-overlapping hyperspherical text prompts for Grounding DINO to improve class-agnostic object detection recall, with angular-coverage-based early termination.
Problem
Class-agnostic object detection with vision-language models suffers low recall when multiple query words are concatenated, because semantically overlapping words reduce detection confidence; manual prompts are impractical for locating all objects.
Approach
Progressively expand learnable prompts by splitting the highest-uncertainty parent prompt into K child prompts via random rotations on the unit hypersphere, using dispersion losses (child-child repulsion, parent-child attraction) and self-training on pseudo-labels, and stop when maximum angular coverage converges.
Key results
- On MS-COCO class-agnostic OD, DiPEx reaches AR100 63.2 and AP 35.9, outperforming CoOp (AR100 61.3, AP 34.6).
- On LVIS, DiPEx achieves AR200 48.4 versus SAM 42.7 (+13.3%) and AP 15.2 versus SAM 6.1 (+21.3% reported in text) and exceeds CoOp by 20.1% in AR200.
- In downstream OOD-OD on MS-COCO, DiPEx reaches unknown-class AR100 59.9 versus 43.3 for zero-shot Grounding DINO with generic prompt.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| expand learnable prompts with DiPEx instead of CoOp prompt tuning | Grounding DINO · MS-COCO val class-agnostic OD | AR@100: +3.1% | Helped |
| expand learnable prompts with DiPEx instead of CoOp prompt tuning | Grounding DINO · MS-COCO val class-agnostic OD, small objects | AR@S: 7.7% | Helped |
| expand learnable prompts with DiPEx instead of SAM | class-agnostic object detection · LVIS val | AP: 21.3% | Helped |
| expand learnable prompts with DiPEx instead of zero-shot Grounding DINO | Grounding DINO · MS-COCO OOD-OD unknown classes | AR@100: 38.3% | Helped |
| merge separate predictions from multiple prompts rather than concatenate the prompts into one query | Grounding DINO with ChatGPT universal words · MS-COCO class-agnostic OD | AR: -52.46% | Helped |
| merge separate predictions from multiple prompts rather than concatenate the prompts into one query | Grounding DINO with WordNet class-wide words · MS-COCO class-agnostic OD | AR: -23.64% | Helped |
| use merged generic and WordNet 25-noun pseudo-labels instead of generic-only pseudo-labels (ablation) | DiPEx · MS-COCO class-agnostic OD | AR@100: 65.5 -> 63.2 | No clear effect |
| use generic-only pseudo-labels instead of WordNet 25-noun-only pseudo-labels (ablation) | DiPEx · MS-COCO class-agnostic OD | AR@100: 46.6 -> 65.5 | Helped |
| increase the number of prompts (ablation) | DiPEx · MS-COCO class-agnostic OD | AR and AP | Helped |
| increase the number of prompts (ablation) | CoCoOp · MS-COCO class-agnostic OD | AR and AP | Hurt |
| increase the child-child dispersion loss coefficient to a large value (γ = 5) (ablation) | DiPEx · MS-COCO class-agnostic OD | detection performance | Hurt |
Surprising findings
- Concatenating semantically overlapping hand-crafted queries into one string reduced AR by up to 52.46% relative to separate-query prediction merging.
- Increasing prompt count helps DiPEx but CoCoOp's performance declines with more prompts.
Limitations
- Requires self-training on the entire dataset for each expansion round, increasing computational overhead.
- Hyperparameters such as temperatures tau_p, tau_c and prompt length K need manual tuning.
- Evaluated only on class-agnostic OD and OOD-OD; open-vocabulary and open-world detection not yet validated.
Experimental setup
Datasets: MS-COCO LVIS PASCAL-VOC
Benchmarks: MS-COCO LVIS
Metrics: AP AR@1 AR@10 AR@100 AR@200 AR@S AR@M AR@L
Baselines: Selective Search UP-DETR DETReg FreeSOLO Exemplar-FreeSOLO MOST CutLER HASSOD CoOp CoCoOp SAM VOS
Builds on: CoOp CoCoOp Grounding DINO MaPLe
Closest papers by meaning
- Just a Few Glances: Open-Set Visual Perception with Image Prompt Paradigm (AAAI 2025 · similarity 0.87)
- DoGA: Enhancing Grounded Object Detection via Grouped Pre-Training with Attributes (AAAI 2025 · similarity 0.87)
- ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.87)
- CP-DETR: Concept Prompt Guide DETR Toward Stronger Universal Object Detection (AAAI 2025 · similarity 0.87)
- Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detection (CVPR 2024 · similarity 0.87)
- Open-Det: An Efficient Learning Framework for Open-Ended Detection (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.