Research map / Vision-language and multimodal / CLIP and zero-shot
UNEM: UNrolled Generalized EM for Transductive Few-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Unrolls a generalized EM (with learnable class-balance and temperature hyperparameters per layer) into a neural network trained on validation tasks for transductive few-shot classification with vision-only and CLIP models.
Problem
Transductive few-shot methods rely on class-balance and related hyperparameters that are grid-searched on validation data, which is suboptimal, computationally costly, and varies across datasets and backbones.
Approach
Generalize EM with two explicit prediction-statistics hyperparameters (class-balance weight omega and temperature T), map each EM iteration to a network layer (learning-to-optimize unrolling), and learn per-layer hyperparameters by minimizing cross-entropy on validation tasks.
Key results
- UNEM-Gaussian reaches 83.7% vs PADDLE 79.2% on mini-ImageNet 20-shot with WRN28-10 (up to ~10% gain in 5-shot with WRN28-10 per the paper's claim), and 78.5% on CUB 5-shot vs PADDLE 71.2%.
- UNEM-Dirichlet improves over EM-Dirichlet by about 4.2% average over 11 CLIP datasets (77.8% vs 73.6%).
- Gains reach 7.5% on challenging datasets with many classes such as FGVC Aircraft, Stanford Cars, and SUN397.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn class-balance and temperature hyperparameters across unrolled layers instead of fixing them in iterative EM | EM-Dirichlet with CLIP · 11 CLIP benchmarks, 4-shot | accuracy: 73.6 -> 77.8 | Helped |
| Learn class-balance and temperature hyperparameters across unrolled layers instead of fixing them in iterative EM | EM-Dirichlet with CLIP · FGVC Aircraft | accuracy: 24.9 -> 30.4 | Helped |
| Learn class-balance and temperature hyperparameters across unrolled layers instead of fixing them in iterative EM | EM-Dirichlet with CLIP · Stanford Cars | accuracy: 73.5 -> 80.0 | Helped |
| Learn class-balance and temperature hyperparameters across unrolled layers instead of fixing them in iterative EM | EM-Dirichlet with CLIP · SUN397 | accuracy: 80.9 -> 88.5 | Helped |
| Learn hyperparameters rather than fixing the class-balance parameter at |Q| | PADDLE with ResNet-18 · tiered-ImageNet 5-shot | accuracy: 45.4 -> 52.3 | Helped |
| Learn hyperparameters rather than fixing the class-balance parameter at |Q| | PADDLE with WRN28-10 · mini-ImageNet 5-shot | accuracy: 62.6 -> 71.6 | Helped |
Surprising findings
- With WRN28-10 features, the original PADDLE does not improve over ResNet-18 features, attributed to a sub-optimal fixed class-balance hyperparameter.
Limitations
- Hyperparameters are learned only on validation tasks, relying on validation data availability
- Evaluated mainly on fine-grained image classification datasets
Experimental setup
Datasets: mini-ImageNet tiered-ImageNet CUB Caltech101 OxfordPets StanfordCars Flowers102 Food101 FGVCAircraft SUN397 DTD EuroSAT
Benchmarks: mini-ImageNet tiered-ImageNet CUB Caltech101 OxfordPets StanfordCars Flowers102 Food101 FGVCAircraft SUN397 DTD EuroSAT
Metrics: accuracy
Baselines: PADDLE TIM ϱ-TIM LaplacianShot ϱ-AM BD-CSPN PT-MAP LR+ICI Baseline EM-Dirichlet Tip-Adapter CoOp
Builds on: PADDLE EM-Dirichlet Algorithm unrolling (learning to optimize) TIM Tip-Adapter
Closest papers by meaning
- Transductive Zero-Shot and Few-Shot CLIP (CVPR 2024 · similarity 0.87)
- Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference (CVPR 2022 · similarity 0.87)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.86)
- Strong Baselines for Parameter-Efficient Few-Shot Fine-Tuning (AAAI 2024 · similarity 0.86)
- BECLR: Batch Enhanced Contrastive Few-Shot Learning (ICLR 2024 · similarity 0.85)
- Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.