Research map / Vision-language and multimodal / CLIP and zero-shot
Audio-visual Generalized Zero-shot Learning the Easy Way
Shentong Mo, Pedro Morgado
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add class embedding optimization (CEO) (ablation) | audio-visual GZSL baseline · VGGSound-GZSL | harmonic mean: 8.77 -> 10.45 | Improved |
| add AVLA supervised contrastive alignment (ablation) | audio-visual GZSL baseline · VGGSound-GZSL | harmonic mean: 8.77 -> 9.69 | Improved |
| add both CEO and AVLA (ablation) | audio-visual GZSL baseline · VGGSound-GZSL | harmonic mean: 8.77 -> 11.55 | Improved |
| add class separability loss to class embedding optimization (ablation) | class embedding optimization · VGGSound-GZSL | harmonic mean: 10.46 -> 11.55 | Improved |
| replace proximity semantic loss with ranking loss (ablation) | class embedding optimization · VGGSound-GZSL | harmonic mean: 11.55 -> 13.10 | Improved |
| use cross-attention similarity instead of cosine similarity (ablation) | AVLA · VGGSound-GZSL | harmonic mean: 10.64 -> 11.55 | Improved |
| use linear or MLP similarity instead of cross-attention (ablation) | AVLA · VGGSound-GZSL | unseen accuracy: 8.68 -> 8.52 (Linear), 8.46 (MLP) | Hurt |
| increase alpha from 0.5 to 0.9 (ablation) | class embedding optimization · VGGSound-GZSL | harmonic mean: 13.10 -> 11.36 | Hurt |
| decrease margin m from 1 to 0 or 0.5 (ablation) | ranking loss · VGGSound-GZSL | harmonic mean: 13.10 -> 12.57 (m=0), 12.65 (m=0.5) | Hurt |
| use L1 or L2 distance instead of cosine distance (ablation) | class embedding optimization · VGGSound-GZSL | harmonic mean: 13.10 -> 12.18 (l2), 11.66 (l1) | Hurt |
| add class embedding optimization with ImageBind encoders (ablation) | ImageBind features · VGGSound-GZSL | harmonic mean: 19.34 -> 26.90 | Improved |
| use stronger pretrained unimodal/paired/unified encoders | Ours · VGGSound-GZSL | harmonic mean: 11.55 -> 26.90 (ImageBind) | Improved |
Closest papers by meaning
- Audio-Visual Generalised Zero-Shot Learning With Cross-Modal Attention and Language (CVPR 2022 · similarity 0.95)
- Temporal and Cross-Modal Attention for Audio-Visual Zero-Shot Learning (ECCV 2022 · similarity 0.93)
- An eye for an ear: zero-shot audio description leveraging an image captioner with audio-visual token distribution matching (NeurIPS 2024 · similarity 0.89)
- TPR: Topology-Preserving Reservoirs for Generalized Zero-Shot Learning (NeurIPS 2024 · similarity 0.88)
- Improving Generalized Zero-Shot Learning by Exploring the Diverse Semantics from External Class Names (CVPR 2024 · similarity 0.88)
- ZeroDiff: Solidified Visual-semantic Correlation in Zero-Shot Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
