Research map / Label-efficient and robust learning / Recognition and long-tail
A Tale of Two Classes: Adapting Supervised Contrastive Learning to Binary Imbalanced Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that supervised contrastive learning degrades on binary imbalanced datasets and proposes two SupCon variants (Supervised Minority and Supervised Prototypes) plus two alignment metrics (SAA, CAC) to diagnose representation collapse.
Problem
SupCon yields collapsed, poorly structured representations on binary imbalanced datasets, where a dominant majority class causes embeddings to collapse, reducing downstream classification accuracy.
Approach
Diagnoses SupCon failure with two new neighborhood-based metrics (sample alignment accuracy and class alignment consistency) and proposes two binary-specific fixes: applying supervision only to the minority class with NT-Xent on the majority, and attracting samples to fixed, opposite-pole class prototypes when their similarity is below 0.5.
Key results
- SupCon balanced accuracy drops from over 90% when balanced to 54-57% at 1% minority on plants and animals, falling below weighted cross-entropy by over 5%.
- Supervised Minority outperforms SupCon by 20% to 35% on iNat21 subsets at imbalance, and Supervised Prototypes achieves 83.4% on plants at 1% minority.
- Class alignment consistency (CAC) achieves overall R2 of 0.69 with linear probing accuracy, versus near-zero for SAD, CAD and GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase class imbalance from 50% to 5% and 1% minority representation | SupCon · Plants, Insects, and Animals | balanced accuracy: 93.7 ± 0.6 -> 56.2 ± 1.6; 54.4 ± 2.0 | Hurt |
| Use Supervised Minority instead of SupCon | binary imbalanced natural image datasets · Plants, Insects, and Animals (5% and 1% minority) | balanced accuracy: 56.2 ± 1.6 -> 89.8 ± 0.6; 54.4 ± 2.0 -> 85.4 ± 0.5 | Helped |
| Use Supervised Prototypes instead of SupCon | binary imbalanced natural image datasets · Plants, Insects, and Animals (5% and 1% minority) | balanced accuracy: 56.2 ± 1.6 -> 88.7 ± 0.7; 54.4 ± 2.0 -> 83.4 ± 1.8 | Helped |
| Use Supervised Minority instead of SupCon | binary imbalanced medical datasets · Infarction, BreastMNIST, PneumoniaMNIST, and FracAtlas fractures | AUC: 61.9 ± 1.9 -> 77.7 ± 1.1; 75.1 ± 0.7 -> 86.4 ± 0.2; 99.5 ± 0.1 -> 99.6 ± 0.1; 84.8 ±0.1 -> 82.3 ±0.7 | Mixed |
| Use Supervised Prototypes instead of SupCon | binary imbalanced medical datasets · Infarction, BreastMNIST, PneumoniaMNIST, and FracAtlas fractures | AUC: 61.9 ± 1.9 -> 77.9 ± 0.4; 75.1 ± 0.7 -> 90.7 ± 0.5; 99.5 ± 0.1 -> 99.8 ± 0.1; 84.8 ±0.1 -> 86.0 ± 0.1 | Helped |
| Use Supervised Minority instead of the strongest long-tailed baseline | binary imbalanced natural image datasets · iNat21 datasets at 5% and 1% minority | balanced accuracy: +1.8; +4.3 | Helped |
| Use Supervised Prototypes instead of the strongest long-tailed baseline | binary imbalanced medical datasets · Infarction (4% minority) | AUC: +2% AUC | Helped |
| Measure local neighborhood class consistency using CAC rather than canonical alignment metrics | representation-space evaluation · all evaluated datasets and methods | R² with linear probing accuracy: 0.045 -> 0.690 | Helped |
Surprising findings
- Canonical alignment and uniformity metrics (SAD, CAD) remain near zero across imbalance levels even as representations collapse.
- SupCon falls below weighted cross-entropy already at around 20% minority class imbalance.
Limitations
- Supervised Minority cannot be applied to balanced data.
- No clear rule for choosing between Supervised Minority and Supervised Prototypes; both should be tested per dataset.
- Supervised Prototypes performed best on medical datasets while Supervised Minority was usually better on natural images, with unclear cause.
Experimental setup
Datasets: iNaturalist21 UK Biobank cardiac BreastMNIST PneumoniaMNIST FracAtlas
Benchmarks: iNaturalist21 binary subsets UKBB cardiac BreastMNIST PneumoniaMNIST FracAtlas
Metrics: balanced accuracy AUC SAD CAD GPU SAA CAC R2
Baselines: Weighted CE SupCon PaCo KCL TSC SBC BCL
Builds on: SupCon NT-Xent alignment and uniformity PaCo TSC KCL BCL
Closest papers by meaning
- BaCon: Boosting Imbalanced Semi-supervised Learning via Balanced Feature-Level Contrastive Learning (AAAI 2024 · similarity 0.88)
- PROTOCOL: Partial Optimal Transport-enhanced Contrastive Learning for Imbalanced Multi-view Clustering (ICML 2025 · similarity 0.86)
- ConR: Contrastive Regularizer for Deep Imbalanced Regression (ICLR 2024 · similarity 0.86)
- Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data (ICML 2025 · similarity 0.85)
- Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching (ICLR 2024 · similarity 0.85)
- Cluster-guided Contrastive Class-imbalanced Graph Classification (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.