Research map / Label-efficient and robust learning / Self-supervised and contrastive
DUEL: Duplicate Elimination on Active Memory for Self-Supervised Class-Imbalanced Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DUEL, a self-supervised learning framework with an active memory that repeatedly evicts the most duplicated stored sample (lowest distinctiveness information) to counter class imbalance in pretraining.
Problem
Self-supervised pretraining on raw, class-imbalanced data overfits to frequent classes, and memory-based contrastive methods reflect intra- and inter-class relations unevenly.
Approach
Derives Hebbian Metric Learning with a distinctiveness term and an upper-bound memory-integrated objective, then uses a Duplicate Elimination policy that replaces the most duplicated memory element with new data, applied on top of MoCoV2 and SimCLR.
Key results
- On CIFAR-10 with rho_max=0.75, D-MoCo reaches 77.99±1.71% linear probing accuracy versus 70.12±1.40% for MoCoV2.
- On STL-10 with rho_max=0.75, D-MoCo reaches 77.23±0.48% versus 75.34±1.11% for MoCoV2.
- Memory class entropy of D-MoCo is higher than MoCo at rho_max=0.75 (1.8306 vs 1.0523).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the DUEL process to MoCoV2 | MoCoV2 · STL-10, class-imbalanced settings | linear probing accuracy: 79.59±6.57 -> 79.20±3.56; 79.32±1.02 -> 76.03±2.06; 77.28±2.68 -> 78.65±2.23; 75.34±1.11 -> 77.23±0.48 | Mixed |
| Add the DUEL process to MoCoV2 | MoCoV2 · CIFAR-10, class-imbalanced settings | linear probing accuracy: 80.99±1.85 -> 82.58±3.48; 82.50±1.12 -> 81.40±2.24; 76.54±1.97 -> 80.53±3.04; 70.12±1.40 -> 77.99±1.71 | Helped |
| Add the DUEL process to SimCLR | SimCLR · STL-10, class-imbalanced settings | linear probing accuracy: 70.80±1.21 -> 75.89±2.90; 69.59±0.50 -> 72.68±3.53; 67.20±1.10 -> 78.23±4.38; 68.75±2.08 -> 74.13±1.04 | Helped |
| Add the DUEL process to SimCLR | SimCLR · CIFAR-10, class-imbalanced settings | linear probing accuracy: 82.28±1.26 -> 82.82±1.10; 78.90±1.60 -> 82.37±1.53; 76.67±1.96 -> 79.56±2.61; 72.81±1.39 -> 75.17±3.49 | Helped |
| Add the DUEL process to MoCo | MoCo · CIFAR-10, ρmax=0.75 | inter-class similarity: 0.0723 -> -0.0513 | Helped |
| Add the DUEL process to MoCo | MoCo · CIFAR-10, ρmax=0.1 | class entropy: 2.2991 -> 2.2988 | No clear effect |
| Add the DUEL process to MoCo | MoCo · CIFAR-10, ρmax=0.5 | class entropy: 1.8394 -> 2.1654 | Helped |
Surprising findings
- Baseline MoCoV2 performs best at mild imbalance on STL-10 and CIFAR-10 (e.g., 82.50% at rho_max=0.25 on CIFAR-10), where D-MoCo is comparable or slightly lower.
Limitations
- Finding the optimal memory management policy is hard; DUEL is an approximation without optimality guarantees.
- Class-imbalance is simulated with one dominant class, and gains are not uniform (MoCoV2 is stronger at some mild imbalance settings).
Experimental setup
Datasets: CIFAR-10 STL-10 ImageNet-LT
Benchmarks: CIFAR-10 STL-10 ImageNet-LT
Metrics: linear probing accuracy class entropy intra-class variance inter-class similarity
Baselines: MoCoV2 SimCLR Barlow Twins BYOL
Builds on: MoCo SimCLR InfoNCE Barlow Twins BYOL
Closest papers by meaning
- Improved Algorithm for Deep Active Learning under Imbalance via Optimal Separation (ICML 2025 · similarity 0.85)
- Gradient Reweighting: Towards Imbalanced Class-Incremental Learning (CVPR 2024 · similarity 0.85)
- Instance-wise Supervision-level Optimization in Active Learning (CVPR 2025 · similarity 0.85)
- Defying Imbalanced Forgetting in Class Incremental Learning (AAAI 2024 · similarity 0.85)
- Learning to Unlearn: Instance-Wise Unlearning for Pre-trained Classifiers (AAAI 2024 · similarity 0.83)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.