Research map / Label-efficient and robust learning / Domain adaptation and generalization
DNA: Improving Few-Shot Transfer Learning with Low-Rank Decomposition and Alignment
Ziyu Jiang, Tianlong Chen, Xuxi Chen, Yu Cheng, Luowei Zhou, Lu Yuan, Ahmed Awadallah, Zhangyang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-supervised alignment on target data to SimCLR (ablation) | SimCLR ResNet-50 · iNaturalist-1k 5-shot | accuracy: 45.0 -> 47.4 | Improved |
| add low-rank plus sparse weight decomposition to Align (ablation) | Align · iNaturalist-1k 5-shot | accuracy: 47.4 -> 48.3 | Improved |
| add semi-supervised distillation to SimCLR | SimCLR · iNaturalist-1k 5-shot | accuracy: 45.0 -> 46.3 | Improved |
| add semi-supervised distillation without pre-training | random-init model · iNaturalist-1k 5-shot | accuracy: 12.7 -> 13.0 | No clear effect |
| add semi-supervised distillation to DnA | DnA · iNaturalist-1k 5-shot | accuracy: 48.3 -> 49.6 | Improved |
| initialize FixMatch from SimCLR pre-training | FixMatch · CIFAR100 4-shot | accuracy: 48.9 -> 49.4 | No clear effect |
| add L2-SP to Align | Align · iNaturalist-1k 5-shot | accuracy: 47.4 -> 47.7 | Improved |
| add Delta+BSS to DnA | DnA · iNaturalist-1k 5-shot | accuracy: 48.3 -> 48.5 | Improved |
| add ImageNet SS pre-trained model before DnA (ablation) | DnA · CIFAR100 4-shot | accuracy: 32.8 -> 53.4 | Improved |
| add ImageNet SS pre-trained model before DnA (ablation) | DnA · Food101 10-shot | accuracy: 44.9 -> 63.3 | Improved |
| fix the sparse mask on S (ablation) | DnA decomposition · iNaturalist-1k 5-shot | accuracy: 47.4 -> 47.9 | Improved |
| also freeze low-rank factor V (ablation) | DnA decomposition · iNaturalist-1k 5-shot | accuracy: 47.9 -> 48.3 | Improved |
Closest papers by meaning
- Self-training For Few-shot Transfer Across Extreme Task Differences (ICLR 2021 · similarity 0.89)
- Boosting Few-Shot Visual Learning With Self-Supervision (ICCV 2019 · similarity 0.87)
- Task-Level Self-Supervision for Cross-Domain Few-Shot Learning (AAAI 2022 · similarity 0.87)
- Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty (NeurIPS 2022 · similarity 0.86)
- Towards Understanding Self-Pretraining for Sequence Classification (NeurIPS 2026 · similarity 0.85)
- Flatten Long-Range Loss Landscapes for Cross-Domain Few-Shot Learning (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
