Research map / Label-efficient and robust learning / Recognition and long-tail
Understanding the Behaviour of Contrastive Loss
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add explicit hard-negative mining to the contrastive loss | Contrastive learning · CIFAR10 | accuracy: 83.27 -> 84.19 | Helped |
| Add explicit hard-negative mining to the contrastive loss | Contrastive learning · CIFAR100 | accuracy: 56.44 -> 57.54 | Helped |
| Add explicit hard-negative mining to the contrastive loss | Contrastive learning · SVHN | accuracy: 95.47 -> 95.26 | Hurt |
| Add explicit hard-negative mining to the contrastive loss | Contrastive learning · ImageNet100 | accuracy: 75.10 -> 74.70 | Hurt |
| Remove hardness-aware weighting by using the simple contrastive loss (ablation) | Ordinary contrastive loss · CIFAR10, CIFAR100, SVHN, ImageNet100 | accuracy: 83.27 -> 74.83; 56.44 -> 39.31; 95.47 -> 70.83; 75.10 -> 48.09 | Hurt |
| Add nearest-neighbor negative selection to the simple contrastive loss (ablation) | Simple contrastive loss · CIFAR10, CIFAR100, SVHN, ImageNet100 | accuracy: 74.83 -> 84.84; 39.31 -> 55.71; 70.83 -> 94.99; 48.09 -> 74.31 | Helped |
| Lower temperature from τ=0.3 to τ=0.07 (ablation) | Ordinary contrastive loss · CIFAR10, CIFAR100, SVHN, ImageNet100 | accuracy: 83.27 -> 79.75; 56.44 -> 51.82; 95.47 -> 92.55; 75.10 -> 71.53 | Hurt |
| Raise temperature from τ=0.3 to τ=0.7 (ablation) | Ordinary contrastive loss · CIFAR10, CIFAR100, SVHN, ImageNet100 | accuracy: 83.27 -> 82.69; 56.44 -> 50.99; 95.47 -> 94.17; 75.10 -> 69.03 | Mixed |
| Raise temperature from τ=0.3 to τ=1.0 (ablation) | Ordinary contrastive loss · CIFAR10, CIFAR100, SVHN, ImageNet100 | accuracy: 83.27 -> 82.21; 56.44 -> 48.33; 95.47 -> 92.07; 75.10 -> 63.57 | Hurt |
| Increase temperature (ablation) | Ordinary contrastive loss · CIFAR10, CIFAR100, SVHN, ImageNet100 | tolerance: 0.04 -> 0.372; 0.10 -> 0.343; 0.032 -> 0.197; 0.093 -> 0.456 | Helped |
| Increase temperature (ablation) | Ordinary contrastive loss · CIFAR10, CIFAR100, SVHN, ImageNet100 | uniformity: 3.86 -> 2.96; 3.86 -> 2.96; 3.88 -> 3.05; 3.917 -> 3.08 | Hurt |
| Use hard contrastive loss instead of ordinary contrastive loss | Contrastive learning · CIFAR10, CIFAR100, SVHN, ImageNet100 | uniformity: 3.60 -> 3.89; 3.60 -> 3.88; 3.65 -> 3.91; 3.693 -> 3.932 | Mixed |
Closest papers by meaning
- Model-Aware Contrastive Learning: Towards Escaping the Dilemmas (ICML 2023 · similarity 0.91)
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere (ICML 2020 · similarity 0.89)
- On the Similarities of Embeddings in Contrastive Learning (ICML 2025 · similarity 0.88)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.87)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.86)
- $\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.