Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Epsilon: Exploring Comprehensive Visual-Semantic Projection for Multi-Label Zero-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Epsilon, a frozen-ViT-B/16 MLZSL framework that combines prompt-based group aggregation of image tokens (GPA) with a global forward propagation module (GFP), trained with a RankNet-style loss, reaching state-of-the-art ZSL/GZSL mAP on NUS-Wide and Open-Images-V4.
Problem
Multi-label zero-shot learning must recognize multiple unseen classes per image; existing attention- and principal-vector methods lose local-global semantic integrity and comprehensiveness of global features.
Approach
Group visual prompts aggregate image tokens into several semantic group vectors via a Transformer encoder and cross-attention (GPA), while a GFP module applies multiple learned softmax-weighted global pooling heads to diversify global semantics; both are concatenated and projected into the word-embedding space.
Key results
- On NUS-Wide ZSL, Epsilon reaches 33.2 mAP vs 29.4 for (ML)^2P-Encoder and 25.9 for SDL.
- On NUS-Wide GZSL, Epsilon reaches 14.7 mAP vs 12.1 for SDL.
- On Open-Images-V4 GZSL, Epsilon reaches 81.0 mAP vs 79.9 for (ML)^2P-Encoder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the GPA module to the backbone-only model (M=8) (ablation) | Epsilon · NUS-Wide ZSL | mAP: 28.6 -> 32.4 | Helped |
| add the GPA module to the backbone-only model (M=8) (ablation) | Epsilon · NUS-Wide GZSL | mAP: 10.8 -> 13.2 | Helped |
| add the GFP module to the backbone-only model (M=8) (ablation) | Epsilon · NUS-Wide ZSL | mAP: 28.6 -> 31.8 | Helped |
| add the GFP module to the backbone-only model (M=8) (ablation) | Epsilon · NUS-Wide GZSL | mAP: 10.8 -> 13.6 | Helped |
| use M=8 instead of M=2 (ablation) | Epsilon backbone-only variant · NUS-Wide ZSL | mAP: 27.9 -> 28.6 | Helped |
| use M=8 instead of M=2 (ablation) | Epsilon backbone-only variant · NUS-Wide GZSL | mAP: 10.6 -> 10.8 | Helped |
| use M=8 instead of M=2 (ablation) | Epsilon with GPA · NUS-Wide ZSL | mAP: 30.7 -> 32.4 | Helped |
| use M=8 instead of M=2 (ablation) | Epsilon with GPA · NUS-Wide GZSL | mAP: 12.5 -> 13.2 | Helped |
| use M=8 instead of M=2 (ablation) | Epsilon with GFP · NUS-Wide ZSL | mAP: 30.5 -> 31.8 | Helped |
| use M=8 instead of M=2 (ablation) | Epsilon with GFP · NUS-Wide GZSL | mAP: 12.8 -> 13.6 | Helped |
| add both GPA and GFP modules rather than using either alone (ablation) | Epsilon · NUS-Wide ZSL | mAP: 32.4 -> 33.2 | Helped |
| add both GPA and GFP modules rather than using either alone (ablation) | Epsilon · NUS-Wide GZSL | mAP: 13.6 -> 14.7 | Helped |
Surprising findings
- Lower precision-type metrics on Open-Images-V4 ZSL are low for all methods because most samples contain few unseen labels.
- Reported ablation shows the GPA module helps ZSL more while GFP helps GZSL more.
Limitations
- Performance is sensitive to the number of group prompts M; too many prompts scatter semantics.
- Open-Images-V4 ZSL precision is low for all methods, and the gains over SDL in some F1 metrics are modest.
- Hardware and compute budget are not reported.
Experimental setup
Datasets: NUS-Wide Open-Images-V4
Benchmarks: NUS-Wide Open-Images-V4
Metrics: mAP top-K precision top-K recall top-K F1
Baselines: CONSE LabelEM Fast0Tag Kim et al. LESA BiAM SDL Gen-MLZSL (ML)^2P-Encoder CLIP-FT
Builds on: LESA BiAM Fast0Tag ViT
Closest papers by meaning
- ZeroMamba: Exploring Visual State Space Model for Zero-Shot Learning (AAAI 2025 · similarity 0.90)
- ZeroDiff: Solidified Visual-semantic Correlation in Zero-Shot Learning (ICLR 2025 · similarity 0.90)
- Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning (CVPR 2024 · similarity 0.89)
- Improving Generalized Zero-Shot Learning by Exploring the Diverse Semantics from External Class Names (CVPR 2024 · similarity 0.89)
- Class Semantic Attribute Perception Guided Zero-Shot Learning (AAAI 2025 · similarity 0.89)
- Visual and Semantic Prompt Collaboration for Generalized Zero-Shot Learning (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.