Research map / Vision-language and multimodal / CLIP and zero-shot
Simple Semantic-Aided Few-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SemFew, which automatically generates detailed class descriptions via WordNet definitions plus GPT-3.5 paraphrasing (Semantic Evolution) and a simple two-layer Semantic Alignment Network that maps visual and semantic features into class prototypes for few-shot classification.
Problem
Few-shot image classification suffers from support samples lacking discriminative features; existing semantic-based methods use brief class names or costly expert attributes and rely on complicated fusion networks.
Approach
Automatically evolve class names into detailed, paraphrased LLM-generated descriptions (WordNet definition expanded by GPT-3.5), then use a simple two-layer network to fuse these semantics with visual features into robust prototypes, so high-quality semantics replace complex fusion modules.
Key results
- SemFew reaches 77.63% 5-way 1-shot on MiniImageNet and SemFew-Trans reaches 82.37% 1-shot on TieredImageNet, outperforming SP-CLIP by 2.16%-6.63% in 1-shot settings.
- SemFew-Trans achieves 84.34% 1-shot on CIFAR-FS and 54.27% 1-shot on FC100 (SP-CLIP 48.53%).
- On cross-domain CUB 1-shot, SemFew reaches 59.07%, 9.25% above the best compared method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SemEvo paraphrased descriptions instead of class names (ablation) | SemFew with CLIP text encoder · MiniImageNet 5-way 1-shot | accuracy: 73.98 -> 77.63 | Helped |
| use SemEvo paraphrased descriptions instead of definitions (ablation) | SemFew with CLIP text encoder · CIFAR-FS 5-way 1-shot | accuracy: 82.67 -> 83.65 | Helped |
| use CLIP rather than BERT to encode semantics (ablation) | SemFew · MiniImageNet 5-way 1-shot | accuracy: 77.63 vs 70.41 | Helped |
| add semantic features to visual features for prototype alignment (ablation) | SemFew · MiniImageNet 5-way 1-shot | accuracy: 66.27 -> 77.63 | Helped |
| add visual features to semantic-only prototype alignment (ablation) | SemFew · MiniImageNet 5-way 1-shot | accuracy: 76.77 -> 77.63 | Helped |
| use the mean vector instead of a clustering-derived prototype (ablation) | SemFew · MiniImageNet 1-shot | accuracy: 77.16 -> 77.63 | Helped |
| use cosine-distance classification instead of linear logistic regression (ablation) | SemFew · FC100 5-shot | accuracy: 57.49 -> 62.79 | Helped |
| use cosine distance instead of Euclidean distance (ablation) | SemFew · MiniImageNet 5-way 1-shot | accuracy: 77.56 -> 77.63 | No clear effect |
| use SemFew-Trans instead of FewTURE | few-shot classification · TieredImageNet 5-way 5-shot | accuracy: 89.96 ± 0.55 -> 89.89 ± 0.52 | Hurt |
| use SemFew instead of AFA | cross-domain few-shot classification · Places 5-shot | accuracy: 76.21 -> 74.70 | Hurt |
| use SemFew instead of the strongest listed baseline | cross-domain few-shot classification · CUB 1-shot | accuracy: 49.82 -> 59.07 | Helped |
| use SemFew instead of SP-CLIP | few-shot classification · CIFAR-FS 5-way 1-shot | accuracy: 82.18 ± 0.40 -> 84.34 ± 0.67 | Helped |
Surprising findings
- Using only semantics (S⇒C) without visual features already gives strong results, e.g. 76.77% 1-shot on MiniImageNet.
- Class names alone (SemFew-Name) underperform the full semantics approach, and CLIP text encoding outperforms BERT encoding for semantics.
Limitations
- Reliance on a closed-source LLM (GPT-3.5-turbo) for semantic generation
- Fusion factor k is manually set and strongly affects accuracy
Experimental setup
Datasets: MiniImageNet TieredImageNet CIFAR-FS FC100 Places CUB
Benchmarks: MiniImageNet TieredImageNet CIFAR-FS FC100 CUB Places
Metrics: accuracy 95% confidence interval
Baselines: ProtoNet MatchNet MAML TADAM CAN FEAT FewTURE SP-CLIP SVAE-Proto AM3
Builds on: ProtoNet FEAT FewTURE SP-CLIP AM3
Closest papers by meaning
- Unlocking the Potential of Pre-trained Vision Transformers for Few-Shot Semantic Segmentation through Relationship Descriptors (CVPR 2024 · similarity 0.89)
- Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.89)
- Envisioning Class Entity Reasoning by Large Language Models for Few-shot Learning (AAAI 2025 · similarity 0.89)
- Class Semantic Attribute Perception Guided Zero-Shot Learning (AAAI 2025 · similarity 0.88)
- VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning (NeurIPS 2025 · similarity 0.88)
- Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.