Research map / Detection and segmentation / Open-set and few-shot detection
Compositional Caching for Training-free Open-vocabulary Attribute Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes COMCA, a training-free open-vocabulary attribute detector that builds a compositional image cache (attribute-object compatibility from CC12M co-occurrence and GPT-3.5 scores, soft attribute labels) to refine CLIP predictions.
Problem
Open-vocabulary attribute detection needs attribute annotations that are costly, ambiguous, and dataset-biased; existing methods are training-based and cache-based approaches scale poorly to the large attribute-object output space.
Approach
Build a scalable cache by sampling objects per attribute from a compatibility distribution (database co-occurrence times LLM compatibility), retrieve images for each pair, and assign soft multi-attribute labels using VLM similarity with normalized scores, then blend cache predictions with CLIP scores at inference.
Key results
- COMCA reaches 27.4 mAP on OVAD and 58.1 on VAW with CLIP ViT-B/32 in the box-given setting, versus 17.0 and 50.0 for zero-shot CLIP.
- COMCA outperforms the training-based LOWA on both benchmarks, e.g., 27.4 vs 18.7 mAP on OVAD.
- COMCA improves average performance across seven VLM backbones by +8.9 mAP on OVAD and +7.0 on VAW.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add COMCA cache refinement to CLIP ViT-B/32 | CLIP ViT-B/32 · OVAD | mAP: +10.6 mAP | Helped |
| Add COMCA cache refinement to CLIP ViT-B/32 | CLIP ViT-B/32 · VAW | mAP: +5.2 | Helped |
| Replace image-based cache with COMCA's compatibility-aware cache | CLIP ViT-B/32 · OVAD | mAP: +10.6 mAP | Helped |
| Replace image-based cache with COMCA's compatibility-aware cache | CLIP ViT-B/32 · VAW | mAP: +5.2 | Helped |
| Add COMCA cache refinement | CLIP ViT-B/32 · OVAD | mAP: +10.7 mAP | Helped |
| Add COMCA cache refinement | CLIP ViT-B/32 · VAW | mAP: +0.6 | Helped |
| Use random cache object sampling instead of attribute-object compatibility priors (ablation) | COMCA cache construction · OVAD | mAP: 17.0 -> 16.7 | Hurt |
| Use random cache object sampling instead of attribute-object compatibility priors (ablation) | COMCA cache construction · VAW | mAP: 50.0 -> 57.5 | Helped |
| Cache every attribute-object combination (ablation) | COMCA cache construction · OVAD | mAP: 17.0 -> 10.6 | Hurt |
| Use database and LLM compatibility scores together rather than database scores alone (ablation) | COMCA cache construction · OVAD | mAP | Helped |
| Use database and LLM compatibility scores together rather than LLM scores alone (ablation) | COMCA cache construction · OVAD | mAP | Helped |
| Use soft labels instead of one-hot labels (ablation) | COMCA · OVAD | mAP: 26.4 -> 27.4 | Helped |
Surprising findings
- Naive random or brute-force caches give little or negative gain over zero-shot CLIP on OVAD (Random -0.3 mAP, Brute force -6.4 mAP).
- Cache-based baselines such as TIP-Adapter and image-based caching often underperform zero-shot CLIP in the box-free setting.
Limitations
- Relies on given target objects and bounding boxes in the main setting; box-free performance is lower (21.5 mAP on OVAD).
- Depends on LLM and database quality, with LLM hallucinations and caption biases acknowledged.
- Hyperparameters (alpha, lambda, beta) are tuned on VAW validation and shared across datasets.
Experimental setup
Datasets: CC12M LAION2B CC3M OVAD VAW
Benchmarks: OVAD VAW
Metrics: mAP head mAP medium mAP tail mAP
Baselines: CLIP TIP-Adapter SuS-X IAP OVAD OvarNet ArtVLM LOWA
Builds on: TIP-Adapter SuS-X CLIP OVAD
Closest papers by meaning
- OW-VAP: Visual Attribute Parsing for Open World Object Detection (ICML 2025 · similarity 0.87)
- The Devil is in the Fine-Grained Details: Evaluating Open-Vocabulary Object Detectors for Fine-Grained Understanding (CVPR 2024 · similarity 0.86)
- Detecting Open World Objects via Partial Attribute Assignment (CVPR 2025 · similarity 0.86)
- Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition (ICLR 2025 · similarity 0.86)
- DoGA: Enhancing Grounded Object Detection via Grouped Pre-Training with Attributes (AAAI 2025 · similarity 0.86)
- GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.