Research map / Label-efficient and robust learning / Self-supervised and contrastive
Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select targets by consistency OR confidence instead of all (ablation) | PACMAC · OfficeHome Cl->Pr DINO | accuracy: 68.6 -> 74.0 | Helped |
| intersect consistency and confidence (AND) rather than union (ablation) | PACMAC · OfficeHome Cl->Pr DINO | accuracy: 74.0 -> 72.2 | Hurt |
| use attention-conditioned masks instead of random masks, k=2 (ablation) | PACMAC · OfficeHome Cl->Pr DINO | accuracy: 72.6 -> 74.0 | Helped |
| use attention-conditioned masks instead of random, k=1 (ablation) | PACMAC · OfficeHome Cl->Pr DINO | accuracy: 68.1 -> 68.6 | Helped |
| add source+target in-domain pretraining (finetune acc), MAE (ablation) | MAE ViT-B · OfficeHome Cl->Pr | accuracy: 60.0 -> 66.2 | Helped |
| add source+target in-domain pretraining (finetune acc), DINO (ablation) | DINO ViT-B · OfficeHome Cl->Pr | accuracy: 66.2 -> 68.9 | Helped |
| use separate domain decoders in MAE S+T pretraining (ablation) | MAE ViT-B · OfficeHome Cl->Pr | accuracy: 60.0 -> 57.4 | Hurt |
| remove S+T pretraining before PACMAC, MAE (ablation) | PACMAC · OfficeHome Cl->Pr | accuracy: 67.1 -> 61.2 | Hurt |
| remove S+T pretraining before PACMAC, DINO (ablation) | PACMAC · OfficeHome Cl->Pr | accuracy: 74.0 -> 72.2 | Hurt |
| masking ratio 90% instead of 75% (ablation) | PACMAC · OfficeHome Cl->Pr | accuracy: 74.0 -> 71.0 | Hurt |
| confidence threshold 75% instead of 50% (ablation) | PACMAC · OfficeHome Cl->Pr | accuracy: 74.0 -> 72.9 | Hurt |
| confidence threshold 25% instead of 50% (ablation) | PACMAC · OfficeHome Cl->Pr | accuracy: 74.0 -> 73.7 | No clear effect |
Closest papers by meaning
- A Closer Look at Self-Supervised Lightweight Vision Transformers (ICML 2023 · similarity 0.88)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.88)
- A Theoretical Analysis of Self-Supervised Learning for Vision Transformers (ICLR 2025 · similarity 0.88)
- Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.86)
- Self-Supervised Learning of Intertwined Content and Positional Features for Object Detection (ICML 2025 · similarity 0.86)
- ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.