Research map / Detection and segmentation / Open-set and few-shot detection

Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers?

NeurIPS 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Slot attention methods · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Probes frozen pretrained ViTs with quadratic similarity probes to test whether patch embeddings encode an "IsSameObject" binding signal, finds it strongest in DINOv2 and weaker in MAE, and ablates it to show it helps segmentation and the DINO loss.

Problem

Whether object binding (grouping patch features into coherent objects) naturally emerges in large pretrained Vision Transformers without explicit object-centric mechanisms such as Slot Attention.

Approach

Defines an IsSameObject predicate on patch embedding pairs and decodes it with symmetric quadratic probes, which outperform linear, pointwise class/identity, and distance baselines; the binding signal lies in a low-dimensional subspace, correlates with attention, and its ablation hurts segmentation and the pretraining loss.

Key results

• Quadratic probe on DINOv2-Large reaches 90.2% IsSameObject accuracy, +17.6 pp over the 72.6% majority baseline.
• DINO, supervised ViT and CLIP show clear binding (+10.3 to +17.6 pp), while MAE ViT-L reaches only 76.3% (+3.7 pp).
• Random shuffling of binding vectors at layer 18 lowers ADE20K segmentation mIoU from 44.14 to 39.20 and raises DINO loss from 0.6182 to 0.6749.

Measured effects reported in the paper

Surprising findings

• Binding peaks in middle layers and then shifts toward class-level grouping, with identical objects becoming indistinguishable in late layers.
• Cosine-similarity and attention-based probes are much weaker than the learned quadratic probe, so feature similarity alone does not explain binding.

Limitations

• Assumes the probe cleanly separates each patch embedding into feature and binding components, a simplification.
• Does not establish a causal link between object binding and downstream task performance.
• Downstream evaluation is limited to segmentation, and only patch-level binding is studied.

Experimental setup

Datasets: ADE20K

Benchmarks: ADE20K

Metrics: probe accuracy segmentation mIoU instance mIoU DINO loss Pearson correlation

Baselines: Linear probe Diagonal quadratic probe Object-class/identity probe Majority baseline Distance-based baseline Cosine similarity probe Self-attention probe

Builds on: Slot Attention DINOv2 linear feature hypothesis

Closest papers by meaning

• Understanding the Limits of Vision Language Models Through the Lens of the Binding Problem (NeurIPS 2024 · similarity 0.90)
• Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.88)
• Beyond Bag-of-Words: Diagnosing Compositional Binding Failures in Vision-Language Models (NeurIPS 2026 · similarity 0.87)
• Mechanisms of Object Localization in Vision-Language Models (CVPR 2026 · similarity 0.87)
• Visual symbolic mechanisms: Emergent symbol processing in Vision Language Models (ICLR 2026 · similarity 0.87)
• Binding Visual Features Point by Point (NeurIPS 2026 · similarity 0.87)