Research map / Label-efficient and robust learning / Self-supervised and contrastive
Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning
Jihai Zhang, Xiang Lan, Xiaoye Qu, Yu Cheng, Mengling Feng, Bryan Hooi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multistage contrastive learning with feature-aware negative sampling and representation concatenation | SimCLR · CIFAR-MNIST (CIFAR) | linear eval accuracy: 0.29 -> 0.87 | Improved |
| add MCL to SimCLR | SimCLR · Trifeature (Shape) | linear eval accuracy: 0.81 -> 1.00 | Improved |
| add MCL to OpenAI CLIP ViT (concat of stages) | CLIP ViT-L-14 · MMVP | average accuracy: 20.0 -> 32.6 | Improved |
| add MCL to OpenAI CLIP ResNet-50 | CLIP ResNet-50 · MMVP | average accuracy: 19.3 -> 24.4 | Improved |
| add cross-stage representation concatenation vs individual stage (ablation) | MCL ViT CLIP · MMVP | average accuracy: best single stage 29.6 -> 32.6 | Improved |
| add MCL to CLIP on Orientation and Direction | CLIP ViT-L-14 · MMVP (O&D) | accuracy: 26.7 -> 6.7 | Hurt |
| reinitialize each stage instead of inheriting previous-stage weights (ablation) | MCL · Trifeature (Shape, tau=0.5) | linear eval accuracy: 0.42 -> 0.75 | Improved |
| increase number of clusters K from 5 to 10 (ablation) | MCL on CIFAR-MNIST · CIFAR-MNIST (CIFAR) | linear eval accuracy | No clear effect |
| use MCL across temperatures 0.1, 0.25, 0.5 (ablation) | SimCLR · CIFAR-MNIST (CIFAR) | linear eval accuracy: tau=0.1 0.29 -> 0.83; tau=0.5 0.10 -> 0.87 | Improved |
| train 4 trainable blocks vs 8 in ViT CLIP MCL (ablation) | MCL ViT CLIP · MMVP | average accuracy: 8 blocks 25.9 -> 4 blocks 28.1 | Improved |
| apply baselines IFM, TS, FD instead of MCL | SimCLR · Trifeature/CelebA/CIFAR-MNIST average | average linear eval accuracy: SimCLR 0.83; IFM 0.82, TS 0.77, FD 0.82 | No clear effect |
| add MCL on ImageNet | SimCLR ResNet34 · ImageNet | linear eval accuracy: 0.456 -> 0.482 | Improved |
Closest papers by meaning
- Decoupled Contrastive Learning (ECCV 2022 · similarity 0.89)
- Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift (ICLR 2024 · similarity 0.89)
- BECLR: Batch Enhanced Contrastive Few-Shot Learning (ICLR 2024 · similarity 0.88)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.88)
- $\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs (ICLR 2025 · similarity 0.87)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
