Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
DeIL: Direct-and-Inverse CLIP for Open-World Few-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeIL, a CLIP-based open-world few-shot learning method that uses a frozen CLIP/CLIPN "direct-and-inverse" pretrainer to denoise support labels, DALL-E augmentation, and a trainable MLP adapter with an inverse-guided contrastive loss.
Problem
Open-world few-shot learning, where support-set labels are noisy (from known or unseen classes) and only one to a few samples per class are available, so standard FSL classifiers degrade badly.
Approach
Splits classification into an inverse step (CLIPN estimates which categories a sample does not belong to, to detect mislabeled support samples) and a direct step (CLIP relabels the flagged samples and drives adapter training), with contrastive learning whose negatives come from the inverse model and whose anchors/positives come from direct predictions.
Key results
- On ImageNet 1-shot with 0.3 noisy label ratio, DeIL reaches 62.28% accuracy, versus 59.99% for CaFo and 51.02% for APE-T in the same setting.
- At a 1.0 noisy label ratio on ImageNet 1-shot, DeIL keeps 62.02% while CoOp drops to 4.57% and CLIP-Adapter to 18.85%.
- Ablation on ImageNet 1-shot shows the inverse identification block adds at least 0.5% and DALL-E augmentation adds about 0.6%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DALL-E data augmentation (ablation) | DeIL · ImageNet 1-shot | accuracy (%): about 0.6 points | Helped |
| add DeIL-Pretrainer (label correction) (ablation) | DeIL · ImageNet 1-shot, noise 0.3 | accuracy (%): at least 0.5 points | Helped |
| add inverse (CLIPN) identification block to pretrainer (ablation) | DeIL-Pretrainer · ImageNet 1-shot, noise 0.3 | accuracy (%): at least 0.5 points | Helped |
| add contrastive learning loss to DeIL-Adapter (ablation) | DeIL-Adapter · ImageNet 1-shot, noise 0.3 | accuracy (%): about 0.1 points | Helped |
| use DeIL-Adapter instead of Tip-Adapter or CLIP-Adapter (ablation) | adapter head · ImageNet 1-shot, noise 0.1-0.9 | accuracy (%): about 1.5 points across noise scenarios (vs Tip-Adapter) | Helped |
| use the Direct-and-Inverse correction instead of direct correction alone (ablation) | DeIL-Pretrainer · ImageNet label correction, noise 0.3 | number of residual noisy 1-shot samples (lower is better): 247 -> 172 at noise 0.3 | Helped |
| remove the adapter (linear probe on CLIP features) (ablation) | DeIL pipeline · ImageNet 1-shot, noise 0.3 | accuracy (%): drops to 17.45 from 62.28 | Hurt |
| DeIL vs. CaFo and Tip-Adapter-F baselines | DeIL · ImageNet 1-shot, noise 0.3 | accuracy (%): +2.29 over CaFo (59.99) | Helped |
Surprising findings
- Standard linear-probe CLIP accuracy falls from 22.29% at 0% noise to 0.08% at 100% noise, while DeIL is nearly flat across noise levels.
Limitations
- Relies on DALL-E generated images whose quality and cost are not analyzed in depth.
- Evaluation uses synthetically injected label noise rather than naturally noisy labels.
Experimental setup
Datasets: ImageNet OxfordPets Caltech101 DTD Food101 Sun397 UCF101 EuroSAT FGVC Flower102 StanfordCars
Benchmarks: ImageNet OxfordPets Caltech101 DTD Food101 Sun397 UCF101 EuroSAT FGVC Flower102 StanfordCars
Metrics: accuracy
Baselines: Linear-probe CLIP CoOp Tip-Adapter-F CLIP-Adapter CALIP-FS CaFo APE-T
Builds on: CLIP CLIPN CaFo Tip-Adapter CLIP-Adapter InfoNCE
Closest papers by meaning
- Collaborative Consortium of Foundation Models for Open-World Few-Shot Learning (AAAI 2024 · similarity 0.87)
- Scaling Few-Shot Learning for the Open World (AAAI 2024 · similarity 0.85)
- Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning (CVPR 2025 · similarity 0.83)
- ImagineFSL: Self-Supervised Pretraining Matters on Imagined Base Set for VLM-based Few-shot Learning (CVPR 2025 · similarity 0.83)
- Demystifying CLIP Data (ICLR 2024 · similarity 0.83)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.