Research map / Vision-language and multimodal / CLIP and zero-shot
Envisioning Class Entity Reasoning by Large Language Models for Few-shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ECER-FSL, which uses LLM-generated class entities filtered by text similarity, plus semantic-guided visual pattern extraction and prototype calibration, to improve few-shot classification.
Problem
Few-shot learning with only one or a few visual samples per class struggles to capture nuanced class features, and existing semantic methods use only abstract category names.
Approach
Generate concrete class entities with an LLM (GPT-4o), filter them by CLIP text similarity to the class name, and use them to guide multi-stage visual pattern extraction (SVPE) and calibrate visual prototypes (PC); the backbone is also pre-trained with image-caption contrastive losses using CogVLM captions.
Key results
- 5-way 1-shot with ResNet-12: 80.34% MiniImageNet, 80.79% TieredImageNet, 85.13% CIFAR-FS, 56.12% FC100, average 75.60%, ahead of Sem-Few (73.65% average).
- Visformer-T backbone reaches 81.14% on MiniImageNet and average 76.58% over four FSL datasets.
- On BSCD-FSL, reports gains of 1.39% to 3.71% over the second-best method across four cross-domain datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add global contrastive loss to CE-only pre-training (ablation) | ECER-FSL pre-training · MiniImageNet and TieredImageNet 5-way 1-shot | accuracy (%): 78.30 ± 0.38 -> 79.21 ± 0.29; 79.10 ± 0.47 -> 79.35 ± 0.62 | Helped |
| Add local contrastive loss (ablation) | ECER-FSL pre-training · MiniImageNet and TieredImageNet 5-way 1-shot | accuracy (%): 79.21 ± 0.29 -> 79.78 ± 0.31; 79.35 ± 0.62 -> 80.32 ± 0.27 | Helped |
| Add an adapter mapping textual features to visual representations (ablation) | ECER-FSL pre-training · MiniImageNet and TieredImageNet 5-way 1-shot | accuracy (%): 79.78 ± 0.31 -> 80.34 ± 0.21; 80.32 ± 0.27 -> 80.79 ± 0.57 | Helped |
| Add SVPE to the fine-tuning stage (ablation) | ECER-FSL · MiniImageNet and TieredImageNet 5-way 1-shot | accuracy (%): 65.39 ± 0.29 -> 78.30 ± 0.38; 68.32 ± 0.78 -> 79.10 ± 0.47 | Helped |
| Add PC to SVPE fine-tuning (ablation) | ECER-FSL · MiniImageNet and TieredImageNet 5-way 1-shot | accuracy (%): 78.30 ± 0.38 -> 79.34 ± 0.33; 79.10 ± 0.47 -> 79.51 ± 0.49 | Helped |
| Add entities to SVPE fine-tuning (ablation) | ECER-FSL · MiniImageNet and TieredImageNet 5-way 1-shot | accuracy (%): 78.30 ± 0.38 -> 79.92 ± 0.27; 79.10 ± 0.47 -> 80.21 ± 0.29 | Helped |
| Add PC to SVPE and entity fine-tuning (ablation) | ECER-FSL · MiniImageNet and TieredImageNet 5-way 1-shot | accuracy (%): 79.92 ± 0.27 -> 80.34 ± 0.21; 80.21 ± 0.29 -> 80.79 ± 0.57 | Helped |
| Increase the number of generated entities, with performance plateauing above 10 (ablation) | ECER-FSL · Few-shot classification | accuracy | Helped |
| Increase the number of SVPE stages from 1 to 4 blocks (ablation) | ECER-FSL · Few-shot classification | accuracy | Helped |
| Use ECER-FSL instead of the strongest reported competitor | ECER-FSL · MiniImageNet, TieredImageNet, CIFAR-FS, and FC100 5-way 1-shot | accuracy (%): 80.34 ± 0.21 vs 77.63 ± 0.63; 80.79 ± 0.57 vs 78.96 ± 0.80; 85.13 ± 0.61 vs 83.65 ± 0.70; 56.12 ± 0.41 vs 54.36 ± 0.71 | Helped |
| Use ECER-FSL instead of the strongest competitor | ECER-FSL · BSCD-FSL: ChestX, ISIC, EuroSAT, CropDisease | accuracy (%): 25.12 vs 23.01; 40.13 vs 36.42; 74.13 vs 70.94; 82.13 vs 80.74 | Helped |
Limitations
- Entity generation relies on hallucination-prone LLM outputs filtered only by a similarity threshold to the class name.
- Entity count plateaus after about 10, indicating limited additional class information.
- Gains reported on standard benchmarks; generality beyond them is not established.
Experimental setup
Datasets: MiniImageNet TieredImageNet CIFAR-FS FC100 ChestX ISIC EuroSAT CropDisease
Benchmarks: MiniImageNet TieredImageNet CIFAR-FS FC100 BSCD-FSL (ChestX, ISIC, EuroSAT, CropDisease)
Metrics: accuracy (%) 95% confidence interval
Baselines: SUN FewTURE FGFL CPEA Meta-AdaM ALFA LastShot SVAE-Proto SP-CLIP SIFT Sem-Few GNN
Builds on: ProtoNet CLIP FEAT Sem-Few
Closest papers by meaning
- Simple Semantic-Aided Few-Shot Learning (CVPR 2024 · similarity 0.89)
- VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning (NeurIPS 2025 · similarity 0.88)
- Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.87)
- Improving Generalized Zero-Shot Learning by Exploring the Diverse Semantics from External Class Names (CVPR 2024 · similarity 0.87)
- Liberating Seen Classes: Boosting Few-Shot and Zero-Shot Text Classification via Anchor Generation and Classification Reframing (AAAI 2024 · similarity 0.87)
- Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.