Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

A Dynamic Learning Method towards Realistic Compositional Zero-Shot Learning

AAAI 2024 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Generative feature synthesis for zero-shot

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Realistic Compositional Zero-Shot Learning (RCZSL), a benchmark on re-labelled MIT-States with GAN-generated art and cartoon domains, and a dynamic visual and semantic modulator method for unseen concepts, compositions, and styles.

Problem

Compositional zero-shot learning assumes test compositions use seen attributes and objects in the same image style, but realistic test images can contain unseen concepts, compositions, and image styles simultaneously, causing domain shift.

Approach

Two dynamic modulators adapt visual features (dynamic convolution with kernel weights predicted from semantic and style statistics) and composition prototypes (attention over low-level visual features) to each input image, to mitigate domain shift from unknown factors.

Key results

• On UT-Zappos (CLIP-free), the method reaches AUC 26.6 and HM 37.7 versus SCEN's 25.2 and 32.0, improving over prior SOTA by nearly 5 points on harmonic mean and AUC.
• With CLIP ViT-L/14 backbone on UT-Zappos, AUC 39.6/HM 52.0 vs CSP's 33.0/46.6 (CSP-based comparison reported in Table 3).
• Ablation on RCZSL (art test domain) shows visual plus semantic modulators together raise HM from 19.7 to 27.5 for the photo-train setting.

Measured effects reported in the paper

Limitations

• Only evaluated on image classification; segmentation and video are left to future work.
• Re-labelled MIT-States and GAN-generated domains may contain label noise and incomplete annotations, causing failure cases.
• Visual and text encoders are kept frozen during training; CLIP-based variant only applies the semantic modulator.

Experimental setup

Datasets: MIT-States UT-Zappos C-GQA

Benchmarks: MIT-States UT-Zappos C-GQA MIT-States-RCZSL

Metrics: AUC harmonic mean seen accuracy unseen accuracy

Baselines: Attop LabelEmbed+ TMN SymNet CompCos CGE Co-CGE DeCa SCEN SCD CANet CLIP

Builds on: CLIP CSP Compositional Zero-Shot Learning baselines (CompCos/SCEN) CycleGAN Object Conditioned Network (OCN)

Closest papers by meaning

• Retrieval-Augmented Primitive Representations for Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.92)
• Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning (CVPR 2024 · similarity 0.91)
• TOMCAT: Test-time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning (NeurIPS 2025 · similarity 0.91)
• Beyond Seen Primitive Concepts and Attribute-Object Compositional Learning (CVPR 2024 · similarity 0.90)
• Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning (AAAI 2024 · similarity 0.90)
• Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning (CVPR 2024 · similarity 0.89)