Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VADS, which makes generative zero-shot learning conditions instance-specific by combining visual-derived prior noise (VDKL) with an updated semantic prototype (VOSU), plugged into CLSWGAN, TF-VAEGAN and FREE.
Problem
Generative zero-shot learning generators are conditioned on Gaussian noise and static predefined semantic prototypes, which do not capture per-instance visual variation and lead to poor generalization to unseen classes (overfitting to seen classes).
Approach
Replace pure Gaussian noise with visual prior noise (a local bias from a visual encoder plus a learnable global prior) and update the semantic prototype per sample via a visual-to-semantic prediction network, concatenating both as a dynamic semantic prototype condition for the generator.
Key results
- VADS achieves GZSL harmonic mean H of 79.3 on AWA2, 55.7 on SUN and 74.3 on CUB with CLSWGAN as the generator.
- Adding VADS to CLSWGAN, TF-VAEGAN and FREE improves average Acc/H, e.g., CLSWGAN+VADS raises CUB H from 65.2 to 74.3.
- Average H gains over generative baselines are reported as 6.4% (SUN), 5.9% (CUB) and 4.2% (AWA2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VDKL visual prior (ablation) | VADS with CLSWGAN · CUB | CZSL accuracy: 84.9 -> 86.8 | Helped |
| add VDKL visual prior (ablation) | VADS with CLSWGAN · AWA2 | CZSL accuracy: 78.1 -> 82.5 | Helped |
| add VOSU semantic prototype updating (ablation) | VADS with CLSWGAN · CUB | GZSL harmonic mean: 70.4 -> 74.3 | Helped |
| add VOSU semantic prototype updating (ablation) | VADS with CLSWGAN · AWA2 | GZSL harmonic mean: 77.0 -> 79.3 | Helped |
| add feature enhancement (ablation) | VADS with CLSWGAN · CUB | CZSL accuracy: 85.1 -> 86.8 | Helped |
| add feature enhancement (ablation) | VADS with CLSWGAN · AWA2 | GZSL harmonic mean: 79.1 -> 79.3 | Helped |
| add contrastive loss Lcon (ablation) | VADS with CLSWGAN · CUB | CZSL accuracy: 85.3 -> 86.8 | Helped |
| add contrastive loss Lcon (ablation) | VADS with CLSWGAN · AWA2 | CZSL accuracy: 79.4 -> 82.5 | Helped |
| add semantic consistency loss Lsc (ablation) | VADS with CLSWGAN · CUB | CZSL accuracy: 86.0 -> 86.8 | Helped |
| add semantic consistency loss Lsc (ablation) | VADS with CLSWGAN · AWA2 | GZSL harmonic mean: 78.7 -> 79.3 | Helped |
| use a prior learned from another domain (SUN) (ablation) | VADS · CUB | GZSL harmonic mean: 74.3 -> 69.3 | Hurt |
| add VADS to CLSWGAN | CLSWGAN · AWA2 | GZSL harmonic mean: 69.3 -> 79.3+10.0 | Helped |
Surprising findings
- Using random Gaussian prior noise with a learnable prior yields only modest gains versus visual prior, while transferring a global prior learned on SUN to CUB hurts performance (H 69.3 vs 72.2 without prior).
Limitations
- Gains on some backbones are small (e.g., TF-VAEGAN on CUB +1.2 H).
- Relies on a fixed ViT-Base feature extractor and per-dataset hyperparameter settings.
- Evaluation limited to three standard benchmarks under the proposed split.
Experimental setup
Datasets: AWA2 SUN CUB
Benchmarks: AWA2 SUN CUB
Metrics: Top-1 accuracy (CZSL) seen accuracy S unseen accuracy U harmonic mean H
Baselines: CLSWGAN TF-VAEGAN FREE CLSWGAN+DSP PSVMA DUET TFVAEGAN+SHIP CE-GZSL ICCE HSVA
Builds on: CLSWGAN TF-VAEGAN FREE DSP PSVMA
Closest papers by meaning
- ZeroDiff: Solidified Visual-semantic Correlation in Zero-Shot Learning (ICLR 2025 · similarity 0.91)
- ZeroMamba: Exploring Visual State Space Model for Zero-Shot Learning (AAAI 2025 · similarity 0.90)
- Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning (CVPR 2024 · similarity 0.90)
- GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencoder (ICML 2025 · similarity 0.89)
- Improving Generalized Zero-Shot Learning by Exploring the Diverse Semantics from External Class Names (CVPR 2024 · similarity 0.88)
- Epsilon: Exploring Comprehensive Visual-Semantic Projection for Multi-Label Zero-Shot Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.