Research map / Label-efficient and robust learning / Self-supervised and contrastive
Dissecting Supervised Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train encoder with supervised contrastive loss (hypersphere-projected features) instead of cross-entropy, then linear classifier | ResNet-18 · CIFAR10 with augmentation | test error %: 6.3 -> 5.7 | Helped |
| Train encoder with supervised contrastive loss instead of cross-entropy | ResNet-18 · CIFAR100 with augmentation | test error %: 27.0 -> 24.9 | Helped |
| Train encoder with supervised contrastive loss instead of cross-entropy (no augmentation) | ResNet-18 · CIFAR10 without augmentation | test error %: 16.4 -> 13.8 | Helped |
| Train encoder with supervised contrastive loss instead of cross-entropy (no augmentation) | ResNet-18 · CIFAR100 without augmentation | test error %: 41.8 -> 41.5 | No clear effect |
| Fix classifier weights to a regular simplex arrangement (CE-fix) instead of learned linear classifier (ablation) | ResNet-18 · CIFAR10 with augmentation | test error %: 6.3 -> 6.2 | No clear effect |
| Fix classifier weights to a regular simplex arrangement (CE-fix) (ablation) | ResNet-18 · CIFAR100 with augmentation | test error %: 27.0 -> 26.3 | Helped |
| Fix classifier weights to simplex arrangement (CE-fix) (ablation) | ResNet-18 · CIFAR10 without augmentation | test error %: 16.4 -> 14.7 | Helped |
| Train with supervised contrastive loss instead of CE: time to fit under increasing label corruption | ResNet-18 on CIFAR10/CIFAR100 · random label experiments | time to fit (iterations) scaling | Hurt |
| Projecting encoder outputs onto a hypersphere with radius 1/sqrt(tau) for SC training | ResNet-18 SC · CIFAR10 and CIFAR100 | cosine similarity of class means to ideal simplex | No clear effect |
| Use a ReLU-based encoder output (standard ResNet) rather than unconstrained features | ResNet-18 trained on CIFAR10 · CIFAR10 class separation | cosine similarity across class means | Hurt |
Closest papers by meaning
- Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss (Student Abstract) (AAAI 2024 · similarity 0.90)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.89)
- Generalization Analysis for Supervised Contrastive Representation Learning under Non-IID Settings (ICML 2025 · similarity 0.88)
- Your contrastive learning problem is secretly a distribution alignment problem (NeurIPS 2024 · similarity 0.88)
- Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses (ICML 2024 · similarity 0.87)
- Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.