AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

RankAlign: Unsupervised Vision-Language Representation Alignment via Rank Transformation

Enzhe Zhao, Marco Fiorucci, Lamberto Ballan

NeurIPS 2026 · Vision-language and multimodal › Cross-modal learning · approach: Contrastive vision-language alignment

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RankAlign, an unsupervised method that aligns unpaired vision and language representations by comparing rank-transformed intra-modality similarity structures.

Problem

Unsupervised alignment of unpaired vision and language model representations; existing methods compare raw intra-modality similarity kernels, which are incommensurate across modalities due to differing scales and densities, leading to ill-conditioned optimization.

Approach

Transform each modality's similarity kernel into ranks before comparison, so that alignment operates on commensurate rank structures rather than raw similarity values, motivated by the Platonic Representation Hypothesis.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace raw kernels with rank-based ordinal kernelsBlindMatch with FAQ solver · CIFAR-100 N=100matching accuracy: 0.04 -> 0.66Improved
replace raw kernels with rank-based kernelsBlindMatch with OT solver · CIFAR-100 N=100matching accuracy: 0.01 -> 0.69Improved
replace raw kernels with rank-based kernelsBlindMatch with FHG solver · CIFAR-100 N=100matching accuracy: 0.15 -> 0.79Improved
rank transformRankAlign · CIFAR-100runtime: FAQ N=100: 1.44 -> 2.70 sHurt
rank transform instead of z-score (ablation)RankAlign · CIFAR-100 N=100matching accuracy: 0.00 -> 0.79Improved
rank transform instead of row-wise ranking (ablation)RankAlign · CIFAR-100 N=100matching accuracy: 0.06 -> 0.79Improved
rank transform instead of quantile normalization (ablation)RankAlign · CIFAR-100 N=50matching accuracy: 0.07 -> 0.98Improved
rank-based alignment instead of BlindMatchRankAlign · LAION pseudo-classmatching accuracy: DINOv2 0.59 -> 0.96; CLIP 0.77 -> 0.68Mixed
rank-based alignment instead of BlindMatchRankAlign · CheXpert subsetmatching accuracy: 0.08 -> 0.65 (DeiT)Improved
increase visual label corruption from 0 to 0.40RankAlign · ImageNet-100 N=100matching accuracy: 74.4 -> 41.5Hurt
increase visual label corruption from 0 to 0.40RankAlign · CIFAR-100 N=100matching accuracy: 78.7 -> 68.9Hurt
compare RankAlign with Procrustes and vanilla GW-OTRankAlign · CIFAR-100 N=100 (CLIP)matching accuracy: 0.03 -> 0.79 (vs GW-OT)Improved

Experimental setup

Baselines: direct similarity kernel alignment methods

Builds on: Platonic Representation Hypothesis

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy