Research map / Vision-language and multimodal / Multimodal LLMs
Exploring Vision-Language Models for Imbalanced Learning
Yidong Wang, Zhuohao Yu, Jindong Wang, Qiang Heng, Hao Chen, Wei Ye, Rui Xie, Xing Xie, Shikun Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add lightweight decoder with Softmax over linear probing (ablation) | CLIP · ImageNet-LT | overall accuracy (%): 67.38 -> 75.05 | Improved |
| add lightweight decoder with imbalanced algorithms (ablation) | CLIP · iNaturalist18 | overall accuracy (%): 5.45 -> 69.82 | Improved |
| add decoder with Softmax instead of CoOp prompt tuning | CLIP · ImageNet-LT | overall accuracy (%): 60.30 -> 75.05 | Improved |
| add decoder with Softmax instead of CoOp prompt tuning | CLIP · Places-LT | overall accuracy (%): 26.10 -> 40.15 | Improved |
| add Balanced Softmax to decoder over plain Softmax (ablation) | CLIP decoder · ImageNet-LT | overall accuracy (%): 75.05 -> 79.03 | Improved |
| add imbalanced algorithm (Disalign) to decoder (ablation) | CLIP decoder · Places-LT | overall accuracy (%): 40.15 -> 48.43 | Improved |
| replace full finetuning with decoder plus imbalanced learning | CLIP · iNaturalist18 | overall accuracy (%): 65.43 -> 73.24 | Improved |
| linear probing instead of zero-shot | CLIP · ImageNet-LT | overall accuracy (%): 70.54 -> 67.38 | Hurt |
| scale backbone from ViT-B16 (85M) to ViT-L14 (443M) | CLIP decoder · all datasets | overall accuracy (%) | Improved |
| use Laion-CLIP (2B data) instead of CLIP (400M) as backbone | CLIP decoder with imbalanced methods · iNaturalist18 | overall accuracy (%): 10% | Hurt |
| add decoder to linear probing | CLIP ViT · ImageNet-LT | GPU memory: 13% to 17% | Hurt |
| use CoOp instead of decoder | CLIP · ImageNet-LT carbon | carbon emitted: 67% | Improved |
Closest papers by meaning
- Why are Visually-Grounded Language Models Bad at Image Classification? (NeurIPS 2024 · similarity 0.89)
- Bridge the Modality and Capability Gaps in Vision-Language Model Selection (NeurIPS 2024 · similarity 0.89)
- The Neglected Tails in Vision-Language Models (CVPR 2024 · similarity 0.88)
- Handling Imbalanced Pseudolabels for Vision-Language Models with Concept Alignment and Confusion-Aware Calibrated Margin (ICML 2025 · similarity 0.88)
- Vision-Language Model Selection and Reuse for Downstream Adaptation (ICML 2025 · similarity 0.87)
- Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
