Research map / Detection and segmentation / Open-set and few-shot detection
Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CDS-CZSL, a CLIP-based three-branch CZSL model whose specificity learner (scored by object-cluster diversity and context) reweights attribute predictions and filters compositions in open-world CZSL.
Problem
Compositional zero-shot learning ignores that attributes differ in specificity (e.g., Sliced vs Red), and open-world CZSL suffers from a huge composition search space.
Approach
Estimate attribute specificity from the diversity of object visual clusters (K-means on attribute-denoised object features), make it context-dependent via a learned specificity network over attribute/object word embeddings, use it as a penalty to refine attribute predictions, and filter compositions by specificity thresholds in OW.
Key results
- In closed-world CZSL, CDS-CZSL reaches AUC 22.4 on MIT-States, 39.5 on UT-Zappos and 11.1 on C-GQA, improving AUC over second-best methods by 1.8, 3.2 and 0.6 points.
- In open-world CZSL with specificity-based filtering, CDS-CZSL achieves AUC 8.5 on MIT-States, 32.3 on UT-Zappos and 2.68 on C-GQA, the best among compared methods.
- Ablation shows the 3-branch structure improves over SPM (e.g., MIT-States CW AUC 19.1 to 21.1) and the specificity learning adds further gains (CDS 22.4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CDS-CZSL instead of the strongest reported baseline | CZSL · MIT-States CW | AUC: 20.6 -> 22.4 | Helped |
| Use CDS-CZSL instead of the strongest reported baseline | CZSL · UT-Zappos CW | AUC: 36.0 -> 39.5 | Helped |
| Use CDS-CZSL instead of the strongest reported baseline | CZSL · C-GQA CW | AUC: 10.5 -> 11.1 | Helped |
| Use CDS-CZSL with filtering instead of the strongest reported baseline | CZSL · MIT-States OW | AUC: 6.9 -> 8.5 | Helped |
| Use CDS-CZSL with filtering instead of the strongest reported baseline | CZSL · UT-Zappos OW | AUC: 30.3 -> 32.3 | Helped |
| Use CDS-CZSL with filtering instead of the strongest reported baseline | CZSL · C-GQA OW | AUC: 2.40 -> 2.68 | Helped |
| Add primitive-wise attribute and object learning to composition-wise learning (ablation) | CDS-CZSL · MIT-States CW | AUC: 19.1 -> 21.1 | Helped |
| Add primitive-wise attribute and object learning to composition-wise learning (ablation) | CDS-CZSL · UT-Zappos OW | AUC: 27.1 -> 29.9 | Helped |
| Add context-based specificity to the 3branch model (ablation) | CDS-CZSL · MIT-States CW | AUC: 21.1 -> 22.0 | Helped |
| Add context-based specificity to the 3branch model (ablation) | CDS-CZSL · UT-Zappos OW | AUC: 29.9 -> 31.0 | Helped |
| Add diversity-driven specificity to the 3branch model (ablation) | CDS-CZSL · MIT-States OW | AUC: 7.4 -> 8.3 | Helped |
| Add diversity-driven specificity to the 3branch model (ablation) | CDS-CZSL · UT-Zappos OW | AUC: 29.9 -> 29.5 | No clear effect |
Surprising findings
- Similarity-based feasibility filtering can retain generic pairs and drop specific valid ones, whereas specificity-based filtering retains fewer but more relevant pairs.
- Specificity-based composition filtering reduced C-GQA feasible pairs from 67293 to 30332 while improving HM and AUC.
Limitations
- Occasionally predicts specific pairs unsupported by image content (e.g., Large-Bucket as Empty).
- Filtering is not unmistakable and can retain specific but invalid pairs such as Pressed-Camera.
- Relative gains on the small UT-Zappos dataset are smaller, possibly because specificity learning is less effective at small scale.
Experimental setup
Datasets: MIT-States UT-Zappos C-GQA
Benchmarks: MIT-States UT-Zappos C-GQA
Metrics: S U HM AUC
Baselines: CGE CompCos Co-CGE SCEN CANet CoT ADE CLIP CoOP CSP HPL DFSP
Builds on: CSP HPL DFSP CLIP SAD-SP
Closest papers by meaning
- Beyond Seen Primitive Concepts and Attribute-Object Compositional Learning (CVPR 2024 · similarity 0.91)
- A Dynamic Learning Method towards Realistic Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.91)
- Retrieval-Augmented Primitive Representations for Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.90)
- Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.89)
- TOMCAT: Test-time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning (NeurIPS 2025 · similarity 0.88)
- LOGICZSL: Exploring Logic-induced Representation for Compositional Zero-shot Learning (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.