AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

Arnas Uselis, Andrea Dittadi, Seong Joon Oh

ICML 2026 · Learning theory and algorithms › Kernels and regression theory · approach: Empirical generalization measures · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use pretrained encoders instead of a random-init encoder (ablation)CLIP/SigLIP/DINO embeddings · PUG-Animal, dSprites, MPI3Dlinear factorization R2: 0.12-0.42 -> 0.4-0.6Improved
higher linear factorization (projected R2) across pretrained encoderslinear probes on frozen embeddings · PUG-Animal, dSprites, MPI3D (10% train combinations)compositional accuracy on unseen combinationsImproved
pretraining versus random initialization for cross-concept orthogonality (ablation)OpenCLIP ViT-L/14 · Pretrained vs random encoderscross-concept cosine similarity: 0.32 -> 0.09-0.12Improved
use cross-entropy loss instead of BCE for minimum embedding dimension (ablation)synthetic linearly compositional model · Synthetic data (Euclidean space)embedding dimension needed: BCE approx 2k vs CE near bound kImproved
use spherical (normalized) embedding geometry instead of Euclidean (ablation)synthetic linearly compositional model · Synthetic dataembedding dimension needed: roughly one extra dimensionHurt
increase number of concepts in synthetic training (ablation)networks trained with classification losses · Synthetic dataemergence of linearity and orthogonalityImproved
zero-shot text-encoder probes instead of linear probes (ablation)CLIP-style models · PUG-Animal, ImageNet-AOR2 vs compositional accuracy trendNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy