Research map / Label-efficient and robust learning / Recognition and long-tail
Can contrastive learning avoid shortcut solutions?
Joshua David Robinson, Li Sun, Ke Yu, kayhan Batmanghelich, Stefanie Jegelka, Suvrit Sra
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add IFM to MoCo-v2 | MoCo-v2 · ImageNet100 | top-1 accuracy: 80.4 -> 81.1 | Improved |
| Add IFM with ε=0.1 | MoCo-v2 · COPDGene, logFEV1pp | R-square: 0.566 -> 0.615 | Improved |
| Add IFM with ε=0.1 | MoCo-v2 · COPDGene, CLE | accuracy: 49.6 -> 48.2 | Hurt |
| Add IFM with ε=0.1 | MoCo-v2 · COPDGene, mMRC | accuracy: 50.4 -> 50.4 | No clear effect |
| Use only the perturbed loss Lε rather than averaging it with standard contrastive loss (ablation) | IFM · Classification tasks | average score: 76.0% vs 75.9% | Hurt |
| Re-normalize embeddings after perturbation (ablation) | IFM-MoCo-v2 · STL10, CIFAR10, CIFAR100 | linear readout accuracy: STL10: 92.9% vs 92.9%; CIFAR10: 92.4% vs 92.2%; CIFAR100: 70.3% vs 70.1% | No clear effect |
| Apply perturbations before normalization (ablation) | IFM-MoCo-v2 · STL10, CIFAR10, CIFAR100 | linear readout accuracy: STL10: 92.9% vs 93.0%; CIFAR10: 92.4% vs 92.0%; CIFAR100: 70.3% vs 70.2% | No clear effect |
| Adjust temperature or hard-negative concentration to make instance discrimination harder | SimCLR · Trifeature | linear readout accuracy across color, shape, and texture | Mixed |
| Use IFM | SimCLR · Trifeature | linear readout accuracy across color, shape, and texture | Improved |
| Use IFM | SimCLR · Robust-feature evaluation across tinyImageNet, STL10, CIFAR10, and CIFAR100 | accuracy | Improved |
| Use IFM rather than SimCLR | SimCLR · Non-robust-feature evaluation across tinyImageNet, STL10, CIFAR10, and CIFAR100 | average accuracy: 55.5% vs 56.7% | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- AdCo — AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations From Self-Trained Negative Adversaries (CVPR 2021) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression (ICML 2023 · similarity 0.89)
- The Power of Contrast for Feature Learning: A Theoretical Analysis (JMLR 2023 · similarity 0.89)
- Intriguing Properties of Contrastive Losses (NeurIPS 2021 · similarity 0.88)
- Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learning (NeurIPS 2023 · similarity 0.87)
- Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning (ECCV 2024 · similarity 0.87)
- An Investigation of Representation and Allocation Harms in Contrastive Learning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
