AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Matryoshka Representation Learning

Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham M. Kakade, Prateek Jain, Ali Farhadi

NeurIPS 2022 · Architectures and efficiency › RNNs and normalization · approach: Dataset distillation and data-efficient training · 38 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train nested multi-dimension losses (MRL) instead of independent fixed-feature modelsResNet50 · ImageNet-1K linear probetop-1 accuracyNo clear effect
train with MRL instead of fixed-feature modelsResNet50 · ImageNet-1K 1-NNaccuracy: up to +2%Improved
train with MRL instead of fixed-feature modelsResNet50 · ImageNet-1K retrievalmAP@10: up to +3%Improved
use weight-tied MRL-E instead of separate classifiers (ablation)ResNet50 MRL · ImageNet-1K linear probetop-1 accuracy: within 1% from 16-dimHurt
use MRL instead of SVD, random features or slimmable netsResNet50 · ImageNet-1Kaccuracy at small dimsImproved
add adaptive classification cascade over nested dimensionsMRL ResNet50 · ImageNet-1Krepresentation size at equal accuracy: ~37 vs 512 dims for 76.3%Improved
use adaptive retrieval (shortlist at 16-dim, rerank at 2048) instead of single-shotMRL ResNet50 · ImageNet-1Kretrieval cost: 128x theoretical, 14x wall-clockImproved
train with MRL instead of fixed featuresResNet50 · ImageNet-Aaccuracy: +0.6%Improved
train with MRL instead of fixed featuresResNet50 · FLUID long-tailnovel-class accuracy: up to +2%Improved
train with MRL instead of fixed featuresResNet50 · few-shot nearest class meanaccuracyNo clear effect
logarithmic instead of uniform granularity spacing (ablation)MRL · ImageNet-1KaccuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy