Research map / Label-efficient and robust learning / Recognition and long-tail
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| directly optimize Lalign and Luniform only instead of contrastive loss | STL-10 AlexNet encoders · STL-10 linear and 5-NN classification | accuracy: +0.69 linear on outputs; +0.14 5-NN on outputs; +0.56 linear on fc7; +0.45 5-NN on fc7 | Helped |
| directly optimize Lalign and Luniform only instead of contrastive loss | NYU-DEPTH-V2 encoders · NYU-DEPTH-V2 depth prediction on conv5 and conv4 | MSE: conv5 0.7014 vs 0.7024 (better); conv4 0.7592 vs 0.7575 (worse) | Mixed |
| directly optimize Lalign and Luniform only instead of contrastive loss | MoCo-based encoders · IMAGENET-100 linear classification | top1 accuracy: +1.80 | Helped |
| directly optimize Lalign and Luniform only instead of contrastive loss | Quick-Thought Vectors encoders · BOOKCORPUS MR and CR classification | accuracy: MR -3.75; CR -2.91 | Hurt |
| finetune with only alignment or only uniformity (vs both) from a suboptimal-temperature contrastive encoder (ablation) | STL-10 encoder (tau=2.5) · STL-10 validation accuracy finetuning trajectories | validation accuracy | Hurt |
| finetune jointly optimizing both alignment and uniformity (ablation) | STL-10 encoder (tau=2.5) · STL-10 validation accuracy finetuning trajectories | validation accuracy | Helped |
| weight ratio between Lalign and Luniform kept not too large (e.g., <4) (ablation) | STL-10 encoders trained with combined losses · STL-10 validation | representation quality | No clear effect |
| Lalign and Luniform metrics as predictors of representation quality (correlation across encoders) | STL-10, NYU-DEPTH-V2, IMAGENET-100, BOOKCORPUS encoders · all four benchmarks | downstream performance | Helped |
Closest papers by meaning
- On the Similarities of Embeddings in Contrastive Learning (ICML 2025 · similarity 0.90)
- On The Inadequacy of Optimizing Alignment and Uniformity in Contrastive Learning of Sentence Representations (ICLR 2023 · similarity 0.90)
- Diversity Is All You Need for Contrastive Learning: Spectral Bounds on Gradient Magnitudes (NeurIPS 2025 · similarity 0.89)
- Your contrastive learning problem is secretly a distribution alignment problem (NeurIPS 2024 · similarity 0.88)
- Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses (ICML 2024 · similarity 0.88)
- $\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.