Research map / Architectures and efficiency / Attention and transformers

Vision Transformers Learn Gestalt-Like Figure-Ground Cues from Natural Images

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Token merging and masked pretraining · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Probes 25 frozen ViTs (supervised, CLIP-like, MIM, self-distillation) with linear probes for figure-ground assignment on natural images and synthetic stimuli isolating surroundedness, convexity, and symmetry.

Problem

Whether ViTs encode generic, human-like shape-based figure-ground cues (surroundedness, convexity, symmetry) rather than relying only on semantic or texture regularities, and where these cues emerge across layers and training objectives.

Approach

Linear probes with position-dependent biases initialized to a spatial prior are fit on frozen patch features of many ViTs; synthetic stimuli with texture, region size and semantics made uninformative isolate individual shape cues, and probes trained on natural MSRA-10K images are tested zero-shot on those stimuli.

Key results

• DeiT-III-L achieves 87.4% IGE on natural images; Perception Encoder-L is lowest at 74.9%.
• Surroundedness probes trained on natural images reach 93.4% IGE zero-shot for BEiT-3-L, close to its 99.6% when trained on the condition.
• Symmetry is decoded near-perfectly without texture (DINOv3-L 99.2% IGE) but only 1-8% IGE with textures for all models.

Surprising findings

• Symmetry is not linearly decodable from textured regions for any model, despite near-perfect untextured performance.
• Self-distillation (DINO) models retain figure-ground information through the final layer, while other objectives show larger drops.

Limitations

• Linear probes may underestimate symmetry information that a more powerful probe could decode.
• Only three shape cues studied; no direct comparison with human behavioral data.
• Small sample size limits significance of pretraining-objective differences.

Experimental setup

Datasets: MSRA-10K DTD Infinite DSprites

Benchmarks: MSRA-10K synthetic surroundedness condition synthetic convexity condition synthetic symmetry condition (textured and untextured)

Metrics: information gain explained (IGE) accuracy

Baselines: spatial prior

Builds on: linear probing Infinite DSprites DTD

Closest papers by meaning

• Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.87)
• The 3D-PC: a benchmark for visual perspective taking in humans and machines (ICLR 2025 · similarity 0.87)
• Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models (NeurIPS 2024 · similarity 0.86)
• Object segmentation from common fate: Motion energy processing enables human-like zero-shot generalization to random dot stimuli (NeurIPS 2024 · similarity 0.86)
• Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models (NeurIPS 2025 · similarity 0.86)
• Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers? (NeurIPS 2025 · similarity 0.86)