AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers

Zachary Shinnick, Liangze Jiang, Hemanth Saratchandran, Damien Teney, Anton van den Hengel

CVPR 2026 · Architectures and efficiency › Attention and transformers · approach: Efficient and robust vision transformers · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add procedural k-Dyck warm-up before image trainingViT-B/16 · ImageNet-1k (from scratch)accuracy (%): 77.49 -> 79.21 (+1.72)Improved
add procedural k-Dyck warm-up instead of default initViT-T/16 · CIFAR-100 (from scratch)accuracy (%): 68.52 -> 71.98Improved
use FractalDB visual warm-up instead of no warm-upViT-T/16 · CIFAR-100 (from scratch)accuracy (%): 68.52 -> 64.61Hurt
add procedural warm-up before ImageNet-1k pretraining, then fine-tuneViT-B/16 · CIFAR-100 (fine-tuned)accuracy (%): 87.54 -> 89.20Improved
replace 1% of pretraining budget with procedural dataViT-B/16 · ImageNet-1kaccuracy (%): 28% fewer image examples for same accuracyImproved
use k-Dyck Shuffle grammar rather than k-Dyck (ablation)procedural warm-up · CIFAR-100 (from scratch)accuracy (%): 71.98 -> 70.11Hurt
use WW grammar rather than k-Dyck (ablation)procedural warm-up · CIFAR-100 (from scratch)accuracy (%): 68.52 -> 66.44Hurt
keep sequence token order intact (vs shuffled tokens) (ablation)procedural warm-up · CIFAR-100 (from scratch)accuracy (%): 67.22 -> 71.98Improved
keep weight structure (vs shuffling all weights) (ablation)procedural warm-up · CIFAR-100 (from scratch)accuracy (%): 69.51 -> 71.98Improved
keep attention weights intact (vs shuffled) (ablation)procedural warm-up · CIFAR-100 (from scratch)accuracy (%): 70.57 -> 71.98Improved
keep MLP weights intact (vs shuffled) (ablation)procedural warm-up · CIFAR-100 (from scratch)accuracy (%): 70.71 -> 71.98Improved
transfer all layers rather than only first 4 (ablation)procedural warm-up · CIFAR-100 (from scratch)accuracy (%): 68.91 -> 71.98Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy