AI Paper Atlas

Research map / Detection and segmentation / CNN features

Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution

Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Peter Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim Alabdulmohsin, Avital Oliver, Piotr Padlewski and 3 more

NeurIPS 2023 · Detection and segmentation › CNN features · approach: Plain and high-resolution ViTs · 16 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use NaViT packing with variable resolution and token dropping instead of fixed-resolution ViT, compute matchedViT-B/32 to L/16 · JFT pretraining / ImageNet-Aaccuracy: 14.7 -> 32.7 (B/32, 1.4x10^11)Improved
NaViT vs ViT on ObjectNet with center crop at high computeNaViT-B/16 · ObjectNetaccuracy: 52.8 -> 50.7 (B/16, 4.8x10^12)Mixed
NaViT backbone instead of ViT for OWL-ViT detectorOWL-ViT-L/14 · LVISAP rare: 17.2 -> 24.3Improved
NaViT instead of ViT for contrastive pretrainingL/14 · ImageNet zero-shotaccuracy: 68.3 -> 72.9Improved
train with variable resolution U(64,Rmax) instead of fixed resolution (ablation)NaViT-B/16 · JFT pretraining, ImageNet evalaccuracyImproved
sample side length rather than area, with truncated normal biased to low resolution (ablation)NaViT resolution sampling · JFT pretrainingaccuracyImproved
sample token drop rate from Beta distribution instead of constant (ablation)NaViT-B/16 · JFT pretrainingaccuracyImproved
resolution-dependent token dropping rate (ablation)NaViT · JFT pretrainingaccuracyImproved
decrease token dropping rate over training (ablation)NaViT-B/16 · ImageNet 10-shotaccuracyImproved
increase token dropping rate over training (ablation)NaViT-B/16 · ImageNet 10-shotaccuracyHurt
use factorized additive positional embeddings instead of learned 2D (Pix2struct) or ViT 1D (ablation)NaViT-B/16 · JFT, evaluated at new resolutionsaccuracyImproved
fine-tune NaViT at variable resolution 64:512 instead of single resolution (ablation)NaViT-B/16 · ImageNet-1kaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy