AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Attentive Multi-Layer Fusion for Vision Transformers

Laure Ciernik, Marco Morik, Lukas Thede, Luca Eyring, Shinichi Nakajima, Zeynep Akata, Lukas Muttenthaler

ICML 2026 · Architectures and efficiency › Attention and transformers · approach: Efficient and robust vision transformers

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fuse CLS and AP tokens from all layers with attentionlinear probe on last-layer CLS · 20 datasets, 9 ViTsaccuracy gain: 5.54 pp average gain over linear probingImproved
use attentive fusion instead of concatenating layers (ablation)multi-layer probe · 20 datasetsaccuracy gainImproved
concatenate intermediate layers with a linear classifier (ablation)linear probe · 20 datasetsaccuracy gainMixed
add more intermediate layers to the fusion pool (ablation)ALF and linear fusion · CLIP-B-16, DINOv2-B-14, ViT-B-16accuracy gainImproved
attend over all last-layer patch tokens (AAT) instead of layer summary tokensALF · GTSRB, SVHN, Stanford Cars, FGVC Aircraftaccuracy gain: GTSRB 13.47 -> 18.02Mixed
use ALF in place of AATlast-layer attentive probe · 20 datasetsstabilityImproved
combine spatial aggregation across patches with hierarchical aggregation across quarterly layers (ablation)ALF · GTSRB, FER2013accuracyImproved
use both CLS and AP tokens instead of CLS-only or AP-only in ALF (ablation)ALF · 19 datasets, 3 base modelsaccuracy gainImproved
use AP tokens only (ablation)ALF · 19 datasetsaccuracy gainMixed
use ALF for tasks near pretraining domainALF · STL-10accuracy gain: 0.04 ± 0.17No clear effect
apply ALF on PCAMALF · PCAMaccuracy gain: 5.32 (linear) vs 2.85 (attentive)Hurt
prune ALF tokens at inference to about half (ablation)ALF · pruning analysisaccuracyMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy