Research map / Label-efficient and robust learning / Recognition and long-tail
Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo
Chaoning Zhang, Kang Zhang, Trung X. Pham, Axi Niu, Zhinan Qiao, Chang D. Yoo, In So Kweon
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace MoCo v2 with dictionary-free SimMoCo using dual temperature | MoCo v2 · CIFAR100 | Top-1 accuracy: 53.28 -> 54.11 | Improved |
| Remove momentum from SimMoCo to form SimCo | SimMoCo · CIFAR100 | Top-1 accuracy: 54.11 -> 58.35 | Improved |
| Remove the dictionary and use dual temperature | MoCo v2 · STL10 | Top-1 accuracy: 81.25 -> 80.56 | Hurt |
| Remove the dictionary and momentum; use dual temperature | MoCo v2 · SVHN | Top-1 accuracy: 47.25 -> 52.37 | Improved |
| Set the inter-anchor temperature higher than the intra-anchor temperature (ablation) | SimMoCo · CIFAR100 | Top-1 accuracy: 49.52 -> 54.11 | Improved |
| Reverse the dual-temperature assignment, making τ α greater than τ β (ablation) | SimMoCo · CIFAR100 | Top-1 accuracy: 54.11 -> 31.28 | Hurt |
| Use a very low scalar temperature, τ β=0.05, rather than τ β=0.5 (ablation) | SimMoCo · CIFAR100 | Top-1 accuracy: 54.07 -> 43.56 | Hurt |
| Replace last-iteration vector keys with earliest dictionary keys (ablation) | MoCo v2 vector component · CIFAR100 | Top-1 accuracy: 53.43 -> 1 | Hurt |
| Replace newest vector keys with randomly sampled keys (ablation) | MoCo v2 vector component · CIFAR100 | Top-1 accuracy: 53.43 -> 44.03 | Hurt |
| Replace newest scalar keys with earliest scalar dictionary keys (ablation) | MoCo v2 scalar component · CIFAR100 | Top-1 accuracy: 53.32 -> 52.13 | Hurt |
| Remove inter-anchor hardness-awareness (ablation) | SimCo · CIFAR100 | Top-1 accuracy: 53.82 -> 58.35 | Improved |
| Remove inter-anchor hardness-awareness (ablation) | SimMoCo · CIFAR100 | Top-1 accuracy: 50.22 -> 54.11 | Improved |
Closest papers by meaning
- On the Similarities of Embeddings in Contrastive Learning (ICML 2025 · similarity 0.86)
- EMC$^2$: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence (ICML 2024 · similarity 0.84)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.84)
- Dynamics-Aware Preference Optimization for Vision-Language Models (CVPR 2026 · similarity 0.84)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.84)
- The Complexity of Finding Local Optima in Contrastive Learning (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
