Research map / Vision-language and multimodal / Human and embodied modeling

Position: The Platonic Representation Hypothesis

ICML 2024 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that neural network representations across architectures, objectives, and modalities are converging toward a shared statistical model of reality (the "platonic representation"), supported by alignment measurements over 78 vision models and 11 LLMs.

Problem

Whether and why representations learned by different AI models (across architectures, training objectives, and data modalities) are becoming increasingly aligned, and what endpoint this convergence reaches.

Approach

Hypothesizes that scaling models, data, and task diversity drives representational convergence toward a shared representation of the underlying reality; formalizes one candidate via PMI kernels of co-occurrence statistics of discrete events observed through bijective observation functions.

Key results

• Across 78 vision models, those solving more VTAB tasks show tighter mutual nearest-neighbor alignment with each other (weak models vary, strong models cluster).
• Language-vision alignment on WIT increases linearly with LLM language-modeling score and with vision model capability; CLIP models align more strongly, though alignment decreases after ImageNet fine-tuning.
• LLM alignment to DINOv2 correlates with downstream Hellaswag performance; the paper also reports alignment scores that remain low in absolute terms (about 0.16 max in Fig. 3).

Measured effects reported in the paper

Surprising findings

• CLIP alignment with LLMs drops after fine-tuning on ImageNet classification.
• Absolute alignment scores remain modest (around 0.16 on the mutual nearest-neighbor metric), leaving much of the gap unexplained.

Limitations

• Argument strictly holds only for bijective observations; information unique to a modality may prevent convergence.
• Convergence is not yet observed uniformly across domains such as robotics.
• Absolute alignment values are low and the metric choice is debated.

Experimental setup

Datasets: Places-365 VTAB WIT (Wikipedia captions) OpenWebText Hellaswag GSM8K CIFAR-10

Benchmarks: VTAB Hellaswag GSM8K

Metrics: mutual nearest-neighbor alignment bits-per-byte transfer accuracy

Baselines: CKA SVCCA

Builds on: model stitching CKA Anna Karenina scenario contrastive learning

Closest papers by meaning

• Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities (NeurIPS 2026 · similarity 0.91)
• Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts (NeurIPS 2025 · similarity 0.88)
• Platonic Task Arithmetic (NeurIPS 2026 · similarity 0.87)
• Platonic Representations in the Human Brain: Unsupervised Recovery of Universal Geometry (NeurIPS 2026 · similarity 0.87)
• Objective drives the consistency of representational similarity across datasets (ICML 2025 · similarity 0.87)
• Towards a learning theory of representation alignment (ICLR 2025 · similarity 0.87)