Research map / Label-efficient and robust learning / Domain adaptation and generalization
Reconstruction Target Matters in Masked Image Modeling for Cross-Domain Few-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes why masked autoencoders underperform on cross-domain few-shot learning and proposes DAMIM, which reconstructs aggregated multi-layer features with a lightweight decoder, reaching SOTA on four CDFSL datasets.
Problem
Masked autoencoders (MAE) transfer poorly to cross-domain few-shot learning under large domain gaps, even underperforming supervised ViT; the paper investigates how the reconstruction target affects transferability.
Approach
Pixel reconstruction makes MAE absorb low-level, domain-specific information, while reconstructing deep high-level features loses global structure. DAMIM reconstructs a learned weighted aggregation of multi-layer auxiliary-encoder features and uses a lightweight one-head decoder with cosine-similarity attention to keep the encoder from over-relying on the decoder.
Key results
- On the 5-way 5-shot setting with ViT-S, DAMIM reaches 65.77% average without fine-tuning and 67.55% with fine-tuning, exceeding FLoR (66.83%) and StyleAdv (66.08%).
- Ablation: AFR alone improves the baseline average from 63.58% to 65.08%, LD alone to 65.24%, and both together to 65.77%.
- Applying the method to MFF raises its 5-shot average from 65.34% to 65.96%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace pixel reconstruction with aggregated multi-layer feature targets and use the Lightweight Decoder (ablation) | MAE for cross-domain few-shot learning · Average, 5-way 5-shot | accuracy: 63.58 -> 65.77 | Helped |
| add Aggregated Feature Reconstruction (AFR) (ablation) | Baseline · Average, 5-way 5-shot | accuracy: 63.58 -> 65.08 | Helped |
| add Lightweight Decoder (LD) (ablation) | Baseline · Average, 5-way 5-shot | accuracy: 63.58 -> 65.24 | Helped |
| remove the MLP and replace query-key attention with cosine similarity (ablation) | Lightweight Decoder · Average, 5-way 5-shot | accuracy: 64.43 -> 65.24 | Helped |
| replace pixel-based MFF with DAMIM reconstruction targets | MFF · Average, 5-way 5-shot | accuracy: 65.34 -> 65.96 | Helped |
| use pixel-based MFF instead of token-based iBOT | MIM for cross-domain few-shot learning · Average, 5-way 5-shot | accuracy: 64.96 -> 65.34 | Helped |
| use high-level features as reconstruction targets rather than pixel targets | MIM for cross-domain few-shot learning · CDFSL target domains | transferability | No clear effect |
Surprising findings
- MAE with pixel reconstruction underperforms even the supervised ViT baseline on several CDFSL target domains.
- Reconstructing high-level, near-output features does not improve transferability.
Limitations
- Evaluated only on ViT-S with a single source dataset (miniImageNet) and four target datasets.
- The aggregation weights rely on layer reconstruction losses, and the trade-off between domain filtering and global structure is shown mainly through empirical analysis.
Experimental setup
Datasets: miniImageNet ImageNet1K CropDisease EuroSAT ISIC2018 ChestX
Benchmarks: CropDisease EuroSAT ISIC2018 ChestX
Metrics: 5-way 1-shot accuracy 5-way 5-shot accuracy domain similarity (CKA)
Baselines: MAE iBOT MFF PMF StyleAdv MEM-FS SDT FLoR MEM-FS+RDA
Builds on: MAE iBOT DINO ProtoNet StyleAdv
Closest papers by meaning
- A Closer Look at the CLS Token for Cross-Domain Few-Shot Learning (NeurIPS 2024 · similarity 0.88)
- ChA-MAEViT: Unifying Channel-Aware Masked Autoencoders and Multi-Channel Vision Transformers for Improved Cross-Channel Learning (NeurIPS 2025 · similarity 0.88)
- Flatten Long-Range Loss Landscapes for Cross-Domain Few-Shot Learning (CVPR 2024 · similarity 0.87)
- Random Registers for Cross-Domain Few-Shot Learning (ICML 2025 · similarity 0.87)
- The Devil is in Low-Level Features for Cross-Domain Few-Shot Segmentation (CVPR 2025 · similarity 0.87)
- Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understanding (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.