AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Attending on Attention ($A^2$): Smaller Self-Supervised ViTs Localize Better Than Larger Ones

Sreehari Rammohan, Huy Ha, Carl Vondrick

NeurIPS 2026 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Invariance-based self-supervised learning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows smaller self-supervised ViTs have attention maps that localize foreground objects better than larger ones, and proposes combining small-model localization with large-model patch representations for robust classification.

Problem

Robust visual classification often requires localizing main foreground objects while ignoring spatially separable distractors; it is unclear how attention localization quality varies with ViT size.

Approach

Smaller self-supervised ViTs yield better foreground-localizing attention than larger ones, while larger ViTs give richer per-patch representations; the method aims to combine good localization from small models with rich features from large models.

Key results

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy