Research map / Detection and segmentation / Open-set and few-shot detection

Beyond Seen Primitive Concepts and Attribute-Object Compositional Learning

CVPR 2024 · Detection and segmentation › Open-set and few-shot detection · approach: Zero-shot segmentation and detection · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Open Vocabulary-Compositional Zero-shot Learning (OV-CZSL) with three new benchmarks and a plug-and-play Neighborhood Expansion Loss that uses external vocabulary neighbors and label smoothing to generalize to unseen attributes, objects, and compositions.

Problem

Compositional zero-shot learning only generalizes to unseen compositions of seen attributes and objects; it cannot handle unseen attributes and objects, so recognizing compositions involving novel primitives remains open.

Approach

Extend CZSL to open vocabulary by expanding the label space with language-embedding neighbors from an external vocabulary, and train with a Neighborhood Expansion Loss that combines label smoothing with label propagation to neighbor compositions, applied to attribute, object and pair embedding spaces.

Key results

• On MIT-States, OADis+NEL raises unseen A*O* accuracy from 1.34 to 3.54 and overall Test@1 AUC to 2.41 vs 1.83 for OADis.
• On C-GQA, the proposed method reaches 12.86 on A*O* accuracy vs 6.17 for OADis, while CompCos keeps the best seen AO accuracy.
• Plugging NEL into CANet improves A*O* on MIT-States from 0.58 to 1.53 and on CLIP fine-tuning from 34.50 to 36.45.

Measured effects reported in the paper

Surprising findings

• Existing CZSL methods like CompCos and SCEN collapse on the fully unseen A*O* split despite strong seen-pair accuracy.
• Using too many neighbors (k=10) degrades unseen performance, since noisy neighbor labels hurt generalization.

Limitations

• Gains on the unseen splits are small in absolute terms (e.g., A*O* on MIT-States reaches only 3.54), and seen-pair accuracy drops relative to CompCos.
• The model cannot discriminate similar concepts such as peel vs slice vs chop; it groups them together.
• Closed-world evaluation only, with neighbor lists built from external captions and GloVe similarity.

Experimental setup

Datasets: MIT-States C-GQA VAW-CZSL Visual Genome Flickr30k COCO-captions Localized Narratives

Benchmarks: MIT-States C-GQA VAW-CZSL

Metrics: AUC harmonic mean top-1 accuracy top-3 accuracy

Baselines: LE CompCos OADis SCEN CANet SEKG TF-VAEGAN CLIP

Builds on: OADis Label Embedder (LE) CompCos label smoothing label propagation

Closest papers by meaning

• LOGICZSL: Exploring Logic-induced Representation for Compositional Zero-shot Learning (CVPR 2025 · similarity 0.92)
• Retrieval-Augmented Primitive Representations for Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.92)
• Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning (CVPR 2024 · similarity 0.91)
• A Dynamic Learning Method towards Realistic Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.90)
• Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.90)
• ProCC: Progressive Cross-Primitive Compatibility for Open-World Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.88)