AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Plug-and-Play Attention Linearization for Pretrained Transformers

Kenan Kassab, Alexey Kashevnik, Ammar Ali, Stamatios Lefkimmiatis

ECCV 2026 · Architectures and efficiency › Attention and transformers · approach: Theory of self-attention dynamics · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Tunable Taylor Expansion (TTE) on top of JPN (ablation)LATTE (CLIP) · MSCOCO Captions retrieval (67% attention layers linearized)text recall: 65.1 -> 77.1Improved
add Joint Post-Squaring Normalization (JPN) on top of TTE (ablation)LATTE (CLIP) · MSCOCO Captions retrieval (67% attention layers linearized)text recall: 71.5 -> 77.1Improved
add JPN and TTE to QT-ViT (ablation)LATTE (CLIP) · MSCOCO Captions retrieval (67% attention layers linearized)text recall: 42 -> 77.1Improved
add TTE only to QT-ViT (ablation)LATTE (SigLip) · MSCOCO Captions retrieval (67% attention layers linearized)text recall: 82.3 -> 86.3Improved
add JPN only to QT-ViT (ablation)LATTE (SigLip) · MSCOCO Captions retrieval (67% attention layers linearized)text recall: 82.3 -> 83.2Improved
LATTE instead of QT-ViT at 67% linearizationRT-DETR · COCO detectionmAP@0.5:0.95: 40.6 -> 46.7Improved
LATTE instead of QT-ViT at 67% linearizationDETR · COCO detectionmAP@0.5: 52.5 -> 59.2Improved
LATTE instead of QT-ViT at 67% linearizationCLIP · MSCOCO Captions retrieval (67% attention layers linearized)image recall: 31.2 -> 59.7Improved
LATTE instead of QT-ViT at 75% linearizationCLIP · SugarCrepe (75% linearized)text recall: 5.30 -> 95.91Improved
LATTE instead of QT-ViT at 54% linearizationCLIP · CIFAR-10 (54% linearized)top-1 accuracy: 86.88 -> 90.56Improved
vary calibration subsample from COCO training set (ablation)LATTE (RT-DETR) · COCO detectionmAP50:95No clear effect
raise layer-selection lambda from 0.1 to 5 or more (ablation)LATTE (RT-DETR) · RT-DETR COCO (67% linearized)mAP50:95: 0.44 -> 0.459Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy