Research map / Label-efficient and robust learning / Domain adaptation and generalization
Revisiting Continuity of Image Tokens for Cross-domain Few-shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that disrupting image-token continuity in ViT hurts source-domain accuracy but barely hurts cross-domain few-shot targets, and proposes ReCIT, a source-training scheme combining warm-up patch shuffling with balanced amplitude shuffling to break token continuity and improve CDFSL.
Problem
Cross-domain few-shot learning (CDFSL) where a ViT pretrained on a general domain must transfer to distant target domains with very few labeled samples; the role of image token continuity in ViT generalization under large domain gaps is unexplored.
Approach
Breaking continuity of image tokens (via patch shuffling and clustered, balanced amplitude shuffling in the frequency domain) during source training forces the model to rely on small within-patch patterns, which transfer better, thereby reducing domain gaps and improving target-domain few-shot accuracy.
Key results
- ReCIT achieves 54.43 average 1-shot accuracy without finetuning and 55.87 with finetuning on ViT-S, exceeding the prior best (AttnTemp 54.12 / 55.39).
- In 5-shot, ReCIT reaches 66.76 average without finetuning and 68.06 with finetuning, versus AttnTemp's 66.62 and 67.61.
- Ablation on 5-shot: baseline average 63.91 rises to 66.76 with warm-up and balanced disruption; the source-domain accuracy drops from 97.78 to 96.33.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add image-token continuity disruption | ViT-S / ReCIT · miniImageNet source domain, 5-shot | accuracy: 97.78 -> 96.33 | Hurt |
| add image-token continuity disruption | ViT-S / ReCIT · CDFSL target-domain average, 5-shot | accuracy: 63.91 -> 66.76 | Helped |
| add warm-up spatial-domain disruption (ablation) | ReCIT · CDFSL 5-shot, average | accuracy: 63.91 -> 65.19 | Helped |
| add balanced frequency-domain disruption after warm-up (ablation) | ReCIT · CDFSL 5-shot, average | accuracy: 65.19 -> 66.76 | Helped |
| remove positional embeddings (ablation) | ViT-S · CDFSL 5-shot, average | accuracy: 63.91 -> 64.81 | Helped |
| shuffle image patches (ablation) | ViT-S · CDFSL 5-shot, average | accuracy: 63.91 -> 64.70 | Helped |
| shuffle patch amplitudes (ablation) | ViT-S · CDFSL 5-shot, average | accuracy: 63.91 -> 65.06 | Helped |
| shuffle patch phases (ablation) | ViT-S · CDFSL 5-shot, average | accuracy: 63.91 -> 64.98 | Helped |
| balance the amplitude shuffling across clusters and patches (ablation) | ViT-S · CDFSL 5-shot, average | accuracy: 65.06 -> 65.94 | Helped |
| apply balanced amplitude shuffling at patch rather than image level (ablation) | ViT-S · CDFSL 5-shot, average | accuracy: 64.61 -> 65.94 | Helped |
| reduce the maintained continuity from clusters to patches (ablation) | ViT-S · CDFSL 5-shot, average | accuracy: 65.44 -> 65.94 | Helped |
Surprising findings
- Disrupting image token continuity hurts source-domain accuracy substantially but only marginally affects target-domain performance, and the amount of source drop correlates with increased source-target CKA similarity.
- Breaking continuity improves target performance even though source-domain accuracy declines (97.78 to 96.33 on miniImageNet).
Limitations
- Evaluated on only four target domains, which may not cover real-world distant domains (acknowledged in Impact Statement).
- Several hyperparameters (similarity threshold, sampling std) are tuned via sensitivity analysis; method relies on hand-designed clustering and disruption schedule.
- Source-domain performance decreases, a trade-off the method accepts.
Experimental setup
Datasets: miniImageNet CropDiseases EuroSAT ISIC2018 ChestX
Benchmarks: CropDiseases EuroSAT ISIC2018 ChestX
Metrics: 5-way 1-shot accuracy 5-way 5-shot accuracy average accuracy CKA similarity
Baselines: MEM-FS StyleAdv FLoR DAMIM CD-CLS AttnTemp PMF MEM-FS + RDA
Builds on: DINO ViT AttnTemp CD-CLS MEM-FS
Closest papers by meaning
- A Closer Look at the CLS Token for Cross-Domain Few-Shot Learning (NeurIPS 2024 · similarity 0.91)
- Random Registers for Cross-Domain Few-Shot Learning (ICML 2025 · similarity 0.91)
- Attention Temperature Matters in ViT-Based Cross-Domain Few-Shot Learning (NeurIPS 2024 · similarity 0.90)
- Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Flatten Long-Range Loss Landscapes for Cross-Domain Few-Shot Learning (CVPR 2024 · similarity 0.87)
- Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinement (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.