Research map / Architectures and efficiency / RNNs and normalization

Cross-Architecture Transferability is Width-Confounded:Diagnosis and Subspace Correction

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that the label-free encoder-selection score RankMe is confounded by penultimate-layer width when used across architectures, and proposes a subspace-based correction.

Problem

Label-free selection of pretrained encoders for downstream transfer; RankMe was validated only within architecture families but is widely applied across architectures, where its rankings may reflect layer width rather than representation quality.

Approach

Diagnoses that cross-architecture RankMe rankings are driven by penultimate-layer dimensionality rather than representational quality, and proposes a subspace correction to remove this width confound.

Key results

• Abstract claims RankMe cross-architecture rankings are systematically driven by penultimate-layer dimension rather than representational quality; further numerical results not stated in the provided abstract.

Surprising findings

• RankMe's cross-architecture rankings are systematically driven by penultimate-layer width rather than representational quality.

Limitations

• Full text unavailable; details of the subspace correction and experiments are unknown.

Experimental setup

Baselines: RankMe

Builds on: RankMe

Closest papers by meaning

• Understanding the Transferability of Representations via Task-Relatedness (NeurIPS 2024 · similarity 0.87)
• TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search (CVPR 2021 · similarity 0.87)
• Implicit Modeling for Transferability Estimation of Vision Foundation Models (NeurIPS 2025 · similarity 0.85)
• No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces (ICML 2025 · similarity 0.85)
• RADAR: Relative Angular Divergence Across Representations (NeurIPS 2026 · similarity 0.84)
• Understanding Task Transfer in Vision-Language Models (CVPR 2026 · similarity 0.84)