Research map / Detection and segmentation / Open-set and few-shot detection
CAKE: Category Aware Knowledge Extraction for Open-Vocabulary Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CAKE, a plug-and-play open-vocabulary detection framework with category-specific knowledge distillation (object- and cluster-level) and a category-generalized RPN that rescores proposals using a CLIP-derived feature set.
Problem
Open-vocabulary object detection suffers from distillation that ignores category-level information and intra-category differences, and from RPNs overfitting to base categories and suppressing novel objects as background.
Approach
Adds a cluster-level distillation branch that aligns per-category feature clusters with CLIP, builds a quality-scored category feature set via SVD-based subspace projection, and uses it to adjust RPN objectness scores so novel-category proposals are retained.
Key results
- On OV-COCO under WS-OVD setting, CAKE reaches 41.8 APnovel50 and 55.7 AP50, versus 36.6 and 49.4 for OC-OVD.
- On OV-LVIS, CAKE achieves 25.0 APr and 34.9 AP detection (versus 21.1 APr and 25.9 AP for OC-OVD), a reported 18.4% gain on APr.
- As a plug-and-play module, CAKE improves OC-OVD, OADP and BARON on OV-COCO, e.g., BARON from 35.8 to 39.4 APnovel50.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CAKE in place of LBP | OV-COCO V-OVD · OV-COCO V-OVD | APnovel50: 37.8 -> 38.2 | Helped |
| Apply CAKE in place of OC-OVD | OV-COCO WS-OVD · OV-COCO WS-OVD | APnovel50: 36.6 -> 41.8 | Helped |
| Apply CAKE in place of LBP | OV-LVIS · OV-LVIS detection | APr: 24.1 -> 25.0 | Helped |
| Add object-level distillation (ablation) | Baseline detector · OV-COCO WS-OVD | APnovel50: 30.4 -> 36.6 | Helped |
| Add cluster-level distillation only (ablation) | Baseline detector · OV-COCO WS-OVD | APnovel50: 30.4 -> 32.5 | Helped |
| Add cluster-level distillation alongside object-level distillation (ablation) | Detector with OD · OV-COCO WS-OVD | APnovel50: 36.6 -> 40.1 | Helped |
| Add CG-RPN to OD + CD (ablation) | Detector with OD + CD · OV-COCO WS-OVD | APnovel50: 40.1 -> 41.3 | Helped |
| Add novel-category text embeddings (ablation) | CAKE · OV-COCO WS-OVD | APnovel50: 41.3 -> 41.8 | Helped |
Surprising findings
- Adding novel category prior knowledge (G-OVD text embeddings) does not significantly improve performance, which the authors attribute to CAKE learning general foreground knowledge from base categories alone.
Limitations
- Does not reach best base-category performance on OV-COCO, attributed to alleviating base overfitting.
- Hyperparameters (lambda weights, temperature) chosen by randomized search on the evaluation datasets.
Experimental setup
Datasets: COCO LVIS v1 OV-COCO OV-LVIS
Benchmarks: OV-COCO OV-LVIS
Metrics: AP50 novel AP50 base AP50 APr APc APf AP mask AP
Baselines: OC-OVD OADP DK-DETR BARON LBP DST-Det GOAT DetPro CORA CoDet RALF Detic
Builds on: OC-OVD Faster R-CNN HRank ViLD
Closest papers by meaning
- SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection (CVPR 2024 · similarity 0.91)
- Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection (ICLR 2025 · similarity 0.91)
- Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detection (CVPR 2024 · similarity 0.90)
- Simple Image-Level Classification Improves Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.90)
- DitHub: A Modular Framework for Incremental Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.89)
- ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.