Research map / Vision-language and multimodal / CLIP and zero-shot

ZeroMamba: Exploring Visual State Space Model for Zero-Shot Learning

AAAI 2025 · Vision-language and multimodal › CLIP and zero-shot · approach: Attribute-based zero-shot learning · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ZeroMamba, a parameter-efficient zero-shot learning framework that adapts VMamba-Small with semantic-aware local projection, global representation learning, and semantic fusion, reporting SOTA on CUB, SUN, AWA2 and ImageNet.

Problem

Zero-shot learning recognizes unseen classes via semantic knowledge, but CNN backbones have limited receptive fields and ViTs have quadratic attention cost, leading to suboptimal visual-semantic interactions.

Approach

Replace CNN/ViT backbones with a Vision Mamba (VMamba-Small) encoder and add semantic-conditioned local attention (SLP), a global semantic MLP (GRL), and an L2-normalized fusion of local and global semantic features (SeF), trained end-to-end with cross-entropy plus an L1 semantic constraint.

Key results

• ZeroMamba reaches 80.0% CZSL accuracy on CUB and 72.4% on SUN, and 74.2 H on CUB GZSL, above prior CNN and ViT methods.
• On ImageNet, ZeroMamba obtains 24.5% zero-shot accuracy versus 23.9% for TransZero++ and 17.6% for I2DFormer+.
• Ablation shows removing L_sc drops CUB CZSL accuracy from 80.0 to 77.6 (4.3 points lower in the paper's text relative to baseline comparison) and the full model is best overall.

Measured effects reported in the paper

Surprising findings

• Removing the semantic constraint loss L_sc hurts CUB GZSL harmonic mean (68.1 vs 74.2 for full model) while raising seen accuracy, showing it mainly regularizes unseen-class performance.
• ZeroMamba-Small gets only 71.9% CZSL on AWA2, below ViT-based ZSLViT-type results on that metric, indicating it is not uniformly best across datasets.

Limitations

• Results rely on dataset-specific hyperparameters (lambda_sc and lambda_col tuned per dataset), limiting robustness claims.
• Not uniformly best: AWA2 CZSL accuracy (71.9%) is competitive rather than top, and some ViT/CNN methods score higher on certain GZSL metrics.
• Experiments use only the Proposed Split and a single VMamba-Small backbone at 448x448 input.

Experimental setup

Datasets: CUB SUN AWA2 ImageNet

Benchmarks: CUB SUN AWA2 ImageNet

Metrics: top-1 accuracy (CZSL) harmonic mean H (GZSL) unseen accuracy U seen accuracy S

Baselines: DAZLE APN TransZero++ HAS ZSLViT DUET I2DFormer+ CLIP LocalVMamba-Small Vim-Small

Builds on: VMamba Vim TransZero DAZLE

Closest papers by meaning

• Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning (CVPR 2024 · similarity 0.93)
• ZeroDiff: Solidified Visual-semantic Correlation in Zero-Shot Learning (ICLR 2025 · similarity 0.92)
• Generalized Zero-Shot Recognition Based on Visually Semantic Embedding (CVPR 2019 · similarity 0.92)
• Epsilon: Exploring Comprehensive Visual-Semantic Projection for Multi-Label Zero-Shot Learning (AAAI 2025 · similarity 0.90)
• Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning (CVPR 2024 · similarity 0.90)
• Visual and Semantic Prompt Collaboration for Generalized Zero-Shot Learning (CVPR 2025 · similarity 0.90)