Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dynamic (EMA) teacher distillation instead of STARTUP's fixed teacher with contrastive loss | ResNet-10 on miniImageNet · BSCD-FSL EuroSAT 5-shot | top-1 accuracy (%): 83.58 -> 89.07 | Helped |
| use dynamic distillation instead of STARTUP | ResNet-10 on miniImageNet · BSCD-FSL ChestX 1-shot | top-1 accuracy (%): 22.27 -> 23.38 | No clear effect |
| use a separate projection head for the distillation loss instead of the shared classifier (ablation) | dynamic distillation network · BSCD-FSL EuroSAT 5-shot | top-1 accuracy (%): 89.07 -> 80.06 | Hurt |
| use DINO-style head with centering and strong augmentation instead of shared classifier head (ablation) | dynamic distillation network · BSCD-FSL EuroSAT 5-shot | top-1 accuracy (%): 89.07 -> 85.74 | Hurt |
| add SimCLR contrastive loss on unlabeled images (ablation) | dynamic distillation network · BSCD-FSL (4 datasets) 5-shot | top-1 accuracy (%): ChestX 28.31 -> 29.45; CropDisease 95.54 -> 93.80 | No clear effect |
| remove supervised base-dataset loss in second stage (ablation) | dynamic distillation network · BSCD-FSL EuroSAT 5-shot | top-1 accuracy (%): 89.07 -> 82.11 | Hurt |
| train jointly from scratch in one step instead of base pretraining then joint training (ablation) | dynamic distillation network · BSCD-FSL CropDisease 5-shot | top-1 accuracy (%): 95.54 -> 87.28 | Hurt |
| use unlabeled data from a different domain than the target (ablation) | dynamic distillation network · BSCD-FSL EuroSAT 5-shot | top-1 accuracy (%): 89.07 -> 81.86 | Hurt |
| use weak-weak or strong-strong augmentation instead of weak-strong (ablation) | dynamic distillation network · BSCD-FSL average 5-shot | top-1 accuracy (%) | Hurt |
| train for more epochs (ablation) | dynamic distillation network · BSCD-FSL average 5-shot | top-1 accuracy (%): +0.2% | No clear effect |
| increase amount of unlabeled data (ablation) | dynamic distillation network · BSCD-FSL average 5-shot | top-1 accuracy (%) | Helped |
| apply method with in-domain unlabeled data | ResNet-18 on tieredImageNet · tieredImageNet 1-shot | top-1 accuracy (%): 61.33 -> 69.00 | Helped |
Closest papers by meaning
- Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification (ICLR 2023 · similarity 0.92)
- Meta-Learning for Semi-Supervised Few-Shot Classification (ICLR 2018 · similarity 0.90)
- Cross-Domain Few-Shot Learning With Task-Specific Adapters (CVPR 2022 · similarity 0.89)
- A Closer Look at Few-shot Classification (ICLR 2019 · similarity 0.89)
- Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learning (CVPR 2024 · similarity 0.88)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.