AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

ThinkingViT: Matryoshka Thinking Vision Transformer for Elastic Inference

Ali Hojjat, Janek Haberer, Sören Pirk, Olaf Landsiedel

CVPR 2026 · Architectures and efficiency › Attention and transformers · approach: Token merging and masked pretraining · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
progressively expand to more attention heads instead of looping the same model (ablation)DeiT-Tiny · ImageNet-1Ktop-1 accuracy (%): 74.10 (naive 2 extra iterations) vs 81.44 (ThinkingViT 3H->6H)Improved
naive looping of same ViT with more iterations (ablation)DeiT-Tiny · ImageNet-1Ktop-1 accuracy (%): 72.20 -> 74.00 -> 74.10 -> 73.60Mixed
add a third thinking stage (12 heads) (ablation)ThinkingViT 3H->6H · ImageNet-1Kaccuracy / GMACs: +0.91 p.p.; 5.85 -> 23.41 GMACsMixed
add a 2H stage before 3H->6H (2H->3H->6H) (ablation)ThinkingViT · ImageNet-1Kaccuracy vs GMACs rangeMixed
add Token Recycling and progressive heads over flat 6-head modelsThinkingViT · ImageNet-1Ktop-1 accuracy (%): up to +2.0 p.p. at equal throughput, +2.9 p.p. at equal GMACsImproved
apply ThinkingViT to a Swin-S backboneSwin-S · ImageNet-1KGMACs at Swin-B accuracy: 15.4 -> 9.5 GMACs at 83.5% accuracyImproved
replace ViT backbone with ThinkingViT in SegmenterSegmenter · ADE20K / CityscapesmIoUImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy