Research map / Label-efficient and robust learning / Recognition and long-tail
When Softmax Fails at the Top: Extreme‑Value Corrections for InfoNCE
Hasan Sabri Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace InfoNCE with WEINCE (anchor-wise Weibull shortfall interpolation) | SimCLR · CIFAR-100 | linear accuracy: +3.27% (ResNet-18), +4.82% (ResNet-50) | Improved |
| replace InfoNCE with WEINCE | SimCLR, ViT-Small · Tiny-ImageNet | linear accuracy: +3.41% | Improved |
| replace InfoNCE with WEINCE | SimCLR, ResNet-18 · Tiny-ImageNet | kNN recall: R@1 +1.55, R@2 +1.35 | Improved |
| replace InfoNCE with WEINCE | SimCSE, BERT-base · STS-B | Spearman x100: 71.74 -> 76.36 | Improved |
| fit a blended endpoint-shortfall link instead of pure softmax link | Frozen SimCLR encoders · CIFAR-10/100, STL-10, ResNet-18/50 | held-out log likelihood per anchor: +0.22 to +0.36 nats | Improved |
| add easier negatives to the choice set (ablation) | Link-selection test · Appendix ablation | blend weight and log-likelihood gain | Hurt |
| increase pool from which a fixed hard set of negatives is selected (ablation) | Link-selection test · Appendix ablation | blend weight lambda | Improved |
| add WEINCE tail statistics to InfoNCE | SimCLR · Tiny-ImageNet/ResNet-18 | step time: 253.3 ms -> 255.0 ms (+0.67%) | No clear effect |
Closest papers by meaning
- Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives (AAAI 2022 · similarity 0.86)
- MultiMax: Sparse and Multi-Modal Attention Learning (ICML 2024 · similarity 0.85)
- Max-Margin Contrastive Learning (AAAI 2022 · similarity 0.84)
- Contrastive Predictive Coding Done Right for Mutual Information Estimation (ICLR 2026 · similarity 0.84)
- Model-Aware Contrastive Learning: Towards Escaping the Dilemmas (ICML 2023 · similarity 0.84)
- Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
