AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning

Jihai Zhang, Xiang Lan, Xiaoye Qu, Yu Cheng, Mengling Feng, Bryan Hooi

ECCV 2024 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Inductive bias and minimal representations · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multistage contrastive learning with feature-aware negative sampling and representation concatenationSimCLR · CIFAR-MNIST (CIFAR)linear eval accuracy: 0.29 -> 0.87Improved
add MCL to SimCLRSimCLR · Trifeature (Shape)linear eval accuracy: 0.81 -> 1.00Improved
add MCL to OpenAI CLIP ViT (concat of stages)CLIP ViT-L-14 · MMVPaverage accuracy: 20.0 -> 32.6Improved
add MCL to OpenAI CLIP ResNet-50CLIP ResNet-50 · MMVPaverage accuracy: 19.3 -> 24.4Improved
add cross-stage representation concatenation vs individual stage (ablation)MCL ViT CLIP · MMVPaverage accuracy: best single stage 29.6 -> 32.6Improved
add MCL to CLIP on Orientation and DirectionCLIP ViT-L-14 · MMVP (O&D)accuracy: 26.7 -> 6.7Hurt
reinitialize each stage instead of inheriting previous-stage weights (ablation)MCL · Trifeature (Shape, tau=0.5)linear eval accuracy: 0.42 -> 0.75Improved
increase number of clusters K from 5 to 10 (ablation)MCL on CIFAR-MNIST · CIFAR-MNIST (CIFAR)linear eval accuracyNo clear effect
use MCL across temperatures 0.1, 0.25, 0.5 (ablation)SimCLR · CIFAR-MNIST (CIFAR)linear eval accuracy: tau=0.1 0.29 -> 0.83; tau=0.5 0.10 -> 0.87Improved
train 4 trainable blocks vs 8 in ViT CLIP MCL (ablation)MCL ViT CLIP · MMVPaverage accuracy: 8 blocks 25.9 -> 4 blocks 28.1Improved
apply baselines IFM, TS, FD instead of MCLSimCLR · Trifeature/CelebA/CIFAR-MNIST averageaverage linear eval accuracy: SimCLR 0.83; IFM 0.82, TS 0.77, FD 0.82No clear effect
add MCL on ImageNetSimCLR ResNet34 · ImageNetlinear eval accuracy: 0.456 -> 0.482Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy