Research map / Vision-language and multimodal / Human and embodied modeling

Characterizing Universal Object Representations Across Vision Models

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Decomposes object-similarity structure of 162 vision models into 50 non-negative NMF dimensions and scores how consistently each recurs across models, finding universal dimensions are more conceptual, interpretable, and predictive of macaque IT and human similarity judgments.

Problem

Prior work shows deep vision networks converge on similar representations, but it is unclear which visual properties they converge on and what factors (architecture, objective, data, size, performance) drive this convergence and whether it relates to biological vision.

Approach

Characterize each model by an RBF similarity matrix over THINGS objects, factorize it with symmetric NMF into non-negative dimensions, and define a null-calibrated, Hungarian-matched cos^2 universality score per dimension across 162 models.

Key results

• Universality scores range from near chance (0.0003) to 0.54; top scores reach 64% of within-model stability ceiling (median ceiling 0.84).
• Universal dimensions are more interpretable: 78% of low-universality dimensions rated 'neither' vs 4% at high universality; semantic+both account for 80% at high universality.
• Universal half of dimensions predicts macaque IT activity better than model-specific half (paired t-test p<.001) and yields higher human triplet accuracy (0.42 vs 0.37, chance 0.33).

Surprising findings

• Universality does not correlate with reconstruction importance (within-model median rho=0.08), so model-specific dimensions contribute as much to similarity structure as universal ones.
• No single inductive bias (architecture, objective, training data), model size, or ImageNet accuracy explains universality.

Limitations

• Model set biased toward systems built for human-useful tasks and may not cover all visual representations.
• Non-negative additive decomposition cannot recover features at multiple abstraction levels or in superposition.
• Only penultimate-layer representations analyzed; other layers may differ.

Experimental setup

Datasets: THINGS ObjectNet macaque IT multi-unit recordings (THINGS images) THINGS odd-one-out triplets

Benchmarks: macaque IT encoding (THINGS MUA) THINGS odd-one-out human triplet accuracy ObjectNet robustness check

Metrics: universality score (null-calibrated cos^2) Spearman correlation Pearson correlation ANOVA omega^2 Hedges g triplet accuracy encoding Pearson r eta^2 category consistency

Baselines: greedy best-match universality CKA

Builds on: symmetric NMF Platonic Representation Hypothesis Hungarian algorithm THINGS odd-one-out embedding (Hebart et al.)

Closest papers by meaning

• Objective drives the consistency of representational similarity across datasets (ICML 2025 · similarity 0.88)
• Representational Similarity via Interpretable Visual Concepts (ICLR 2025 · similarity 0.87)
• Measuring Per-Unit Interpretability at Scale Without Humans (NeurIPS 2024 · similarity 0.87)
• When does perceptual alignment benefit vision representations? (NeurIPS 2024 · similarity 0.86)
• Discriminating image representations with principal distortions (ICLR 2025 · similarity 0.86)
• Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts (NeurIPS 2025 · similarity 0.86)