Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Retrieval-Augmented Primitive Representations for Compositional Zero-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a retrieval-augmented CLIP-based method for compositional zero-shot learning that retrieves same-attribute and same-object training image representations from two dynamically updated databases to enrich primitive representations.
Problem
Compositional zero-shot learning must recognize unseen attribute-object compositions from seen ones, which is hard due to attribute-object entanglement and long-tailed primitive distributions.
Approach
Build attribute and object databases of training-image representations, refreshed each epoch, and use non-parametric top-K cosine retrieval to fuse retrieved same-primitive representations into the query's attribute and object representations, with a de-bias loss and a retrieval loss to make retrieval more accurate.
Key results
- Closed-world UT-Zappos AUC 44.5, HM 56.5 vs. Troika 41.7/54.6 and PLID 38.7/52.4 (CLIP-based).
- Closed-world MIT-States AUC 22.5 and HM 39.2 vs. Troika 22.1/39.3, slightly ahead on AUC.
- Open-world C-GQA AUC 4.4 and HM 14.6 vs. Troika 2.7/10.9; open-world MIT-States AUC 8.18 vs. Troika 7.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use retrieval-augmented primitive representations | CZSL with CLIP ViT-L/14 · MIT-States closed-world | AUC: 22.1 -> 22.5 | Helped |
| Use retrieval-augmented primitive representations | CZSL with CLIP ViT-L/14 · C-GQA closed-world | AUC: 12.4 -> 14.4 | Helped |
| Use retrieval-augmented primitive representations | CZSL with CLIP ViT-L/14 · UT-Zappos closed-world | AUC: 41.7 -> 44.5 | Helped |
| Use retrieval-augmented primitive representations | CZSL with CLIP ViT-L/14 · MIT-States open-world | AUC: 7.3 -> 8.18 | Helped |
| Use retrieval-augmented primitive representations | CZSL with CLIP ViT-L/14 · C-GQA open-world | AUC: 2.7 -> 4.4 | Helped |
| Use retrieval-augmented primitive representations | CZSL with CLIP ViT-L/14 · UT-Zappos open-world | AUC: 33.0 -> 33.3 | Helped |
| Add the retrieval module (ablation) | UT-Zappos CZSL baseline · UT-Zappos closed-world | AUC: 39.4 -> 40.6 | Helped |
| Add the de-bias loss (ablation) | UT-Zappos CZSL baseline · UT-Zappos closed-world | AUC: 39.4 -> 40.4 | Helped |
| Add the de-bias loss to the retrieval-module variant (ablation) | UT-Zappos CZSL · UT-Zappos closed-world | AUC: 40.6 -> 44.0 | Helped |
| Add the retrieval loss to the retrieval-module variant (ablation) | UT-Zappos CZSL · UT-Zappos closed-world | AUC: 40.6 -> 41.0 | Helped |
| Add the retrieval loss to the retrieval-module-plus-de-bias-loss variant (ablation) | UT-Zappos CZSL · UT-Zappos closed-world | AUC: 44.0 -> 44.5 | Helped |
Surprising findings
- The gain over prior methods is larger in the open-world setting than in the closed-world setting, which the authors attribute to needing disentangled, composable primitive representations.
Limitations
- Attribute retrieval fails when the object word is ambiguously used as an attribute (e.g., 'computer' as attribute), per the failure case.
- Requires per-epoch updates of the databases by re-extracting visual representations of selected training images.
- Hyperparameters (K, N_D, M, beta, lambdas) are set separately per dataset.
Experimental setup
Datasets: MIT-States UT-Zappos C-GQA
Benchmarks: MIT-States UT-Zappos C-GQA
Metrics: AUC HM Seen accuracy Unseen accuracy
Baselines: CompCos OADis CGE Co-CGE CANet CAPE CSP DFSP DRPT Troika PLID KG-SP
Builds on: CLIP CSP Troika retrieval-augmented classification (RAC)
Closest papers by meaning
- A Dynamic Learning Method towards Realistic Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.92)
- Beyond Seen Primitive Concepts and Attribute-Object Compositional Learning (CVPR 2024 · similarity 0.92)
- Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning (CVPR 2024 · similarity 0.90)
- Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.89)
- LOGICZSL: Exploring Logic-induced Representation for Compositional Zero-shot Learning (CVPR 2025 · similarity 0.89)
- Beyond Image Classification: A Video Benchmark and Dual-Branch Hybrid Discrimination Framework for Compositional Zero-Shot Learning (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.