Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
From Channel Bias to Feature Redundancy: Uncovering the “Less is More” Principle in Few-Shot Learning
Jie Zhang, Xu Luo, Lianli Gao, Difan Zou, Heng Tao Shen, Jingkuan Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply simple channel-wise transformation phi_k with k=1.3 at test time | few-shot classifiers across backbones · Meta-Dataset/BSCD-FSL/DomainNet 5-way 5-shot | accuracy: 0.5-7.5% on average | Improved |
| apply simple transformation when train and test distributions are very similar | ImageNet-trained models · miniImageNet and MSCOCO test | accuracy | Hurt |
| replace simple transformation with oracle MMC (ablation) | PN NCC · binary 5-shot tasks (avg of 10 datasets) | accuracy: 83.1 -> 87.7 | Improved |
| replace simple transformation with oracle MMC (ablation) | S2M2 linear classifier · binary 5-shot tasks (avg of 10 datasets) | accuracy: 87.8 -> 91.7 | Improved |
| add simple transformation to unadapted features (ablation) | PN NCC · binary 5-shot tasks (avg of 10 datasets) | accuracy: 80.7 -> 83.1 | Improved |
| apply simple transformation to a fine-tuned feature extractor (ablation) | fine-tuning on miniImageNet shots · miniImageNet shot analysis | accuracy | No clear effect |
| increase number of shots for linear classifier with transformation (ablation) | linear classifier · miniImageNet shot analysis (up to 400 shots) | accuracy gain | Mixed |
| mask unimportant feature dimensions, keeping about 110 (linear probing) or 255 (finetuning) (ablation) | ImageNet ResNet-50 features · Aircraft 5-way 1-shot | test error: around 110 for linear probing and 255 for finetuning | Improved |
| add redundant second dimension to features (ablation) | 1-shot LP/NCC · two-dimensional Gaussian toy | accuracy: 90.83 -> 75.87 | Hurt |
| add second dimension with 500 shots (ablation) | linear probing · two-dimensional Gaussian toy | accuracy: 95.20 -> 97.36 | Improved |
| add second dimension with 500 shots (ablation) | NCC · two-dimensional Gaussian toy | accuracy: 95.21 -> 84.37 | Hurt |
| estimate feature importance from 5 randomly cropped augmented views per support image (ablation) | feature importance estimation · few-shot tasks | estimation variance and accuracy | Improved |
Closest papers by meaning
- Channel Importance Matters in Few-Shot Image Classification (ICML 2022 · similarity 0.91)
- Dense Classification and Implanting for Few-Shot Learning (CVPR 2019 · similarity 0.89)
- DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation (NeurIPS 2023 · similarity 0.88)
- Adversarially Robust Few-shot Learning via Parameter Co-distillation of Similarity and Class Concept Learners (CVPR 2024 · similarity 0.87)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.87)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
