Research map / Detection and segmentation / Open-set and few-shot detection
ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ProxyDet, which synthesizes proxy-novel class embeddings by class-wise mixup of quality-weighted base-class prototypes in CLIP space and trains an extra detector head with a proxy L1 loss to improve novel-class detection.
Problem
Open-vocabulary object detection generalizes poorly to novel classes that were never pseudo-labeled from external data, since pseudo-labeling mainly helps the pseudo-labeled novel classes.
Approach
Observes that many novel-class CLIP text embeddings lie within the convex hull of base-class embeddings, and synthesizes proxy-novel classes by linearly mixing pairs of base-class prototypes (visual prototypes weighted by objectness/IoU quality and CLIP text embeddings), then trains a separate head with an L1 proxy loss pulling region embeddings toward mixed text embeddings.
Key results
- On OV-LVIS with ResNet50 and LVIS-base + IN-L, ProxyDet reaches 26.2 AP_r versus Detic's 24.6; with Swin-B it reaches 36.7 vs 33.8.
- On OV-COCO, ProxyDet achieves 30.4 AP_novel, improving over Detic (27.8) by +2.6.
- Cross-dataset evaluation trained on LVIS base reports 57.0 AP50 on COCO, the best among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add proxy-novel class training via class-wise mixup | ProxyDet · LVIS novel classes | AP_r: +1.6 AP_r | Helped |
| Add proxy-novel class training via class-wise mixup | ProxyDet · COCO novel classes | AP_novel: +2.6 | Helped |
| Use instance-wise instead of class-wise mixup (ablation) | ProxyDet · LVIS | AP_r: 24.6 -> 24.8 | No clear effect |
| Use class-wise mixup with visual prototypes instead of instance-wise mixup (ablation) | ProxyDet · LVIS | AP_r: 24.6 -> 25.6 | Helped |
| Weight visual prototypes by IoU (ablation) | ProxyDet · LVIS | AP_r: 25.6 -> 26.0 | Helped |
| Weight visual prototypes by objectness score (ablation) | ProxyDet · LVIS | AP_r: 25.6 -> 26.2 | Helped |
| Sample the mix coefficient from Beta(1, 1) instead of Bernoulli(0.5) (ablation) | ProxyDet · LVIS | AP_r: 24.8 -> 26.2 | Helped |
| Also apply class-wise mixup to IN-L (ablation) | ProxyDet · LVIS novel classes | AP_r: 26.2 -> 23.6 | Hurt |
| Use novel-nearest rather than random base-class selection for mixing (ablation) | ProxyDet · LVIS | AP_r: 26.2 -> 26.9 | Helped |
| Replace L1 proxy loss with cosine-similarity loss (ablation) | ProxyDet · LVIS | AP_r: 26.2 -> 25.3 | Hurt |
| Replace L1 proxy loss with L2 distance (ablation) | ProxyDet · LVIS | AP_r: 26.2 -> 25.1 | Hurt |
| Add pseudo-labeling | OVOD detector · LVIS novel non-pseudo classes | AP_r: 14.2 -> 11.6 (-2.6) | Hurt |
Surprising findings
- Applying class-wise mixup on ImageNet-LVIS, which uses max-size RPN proposals without box labels, degrades novel-class AP_r from 26.2 to 23.6.
- Instance-wise mixup without class-wise prototypes gives essentially no gain (+0.2 AP_r), and Bernoulli(0.5) mixing also yields no gain.
Limitations
- Scaling the mixup to datasets without precise box annotations (ImageNet-LVIS) hurts novel-class performance because pseudo boxes are too coarse.
- Selecting base classes nearest to known novel classes gives better results, but novel-class names are unavailable in the realistic setting, so random mixing is used.
- Gains are reported mainly on LVIS rare and COCO novel splits; the method relies on the quality of region proposals for prototypes.
Experimental setup
Datasets: LVIS ImageNet-LVIS COCO COCO Caption Objects365
Benchmarks: OV-LVIS OV-COCO Objects365
Metrics: AP_r AP AP_novel AP_base AP50 AP50 rare
Baselines: ViLD RegionCLIP F-VLM Detic WSDDN Cap2Det OVR-CNN DetPro
Builds on: Detic CLIP mixup CenterNet2
Closest papers by meaning
- SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection (CVPR 2024 · similarity 0.90)
- Simple Image-Level Classification Improves Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.89)
- CAKE: Category Aware Knowledge Extraction for Open-Vocabulary Object Detection (AAAI 2025 · similarity 0.89)
- Open-Det: An Efficient Learning Framework for Open-Ended Detection (ICML 2025 · similarity 0.89)
- DitHub: A Modular Framework for Incremental Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.89)
- Generative Region-Language Pretraining for Open-Ended Object Detection (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.