Research map / Label-efficient and robust learning / Self-supervised and contrastive

Discovering Global False Negatives On the Fly for Self-supervised Contrastive Learning

ICML 2025 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Contrastive versus non-contrastive theory

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GloFND, an optimization-based method that learns per-anchor thresholds on the fly to detect and filter global false negatives in contrastive learning, integrated with SogCLR, FastCLIP, and SimCLR.

Problem

In self-supervised contrastive learning, negatives sampled from the whole dataset often include semantically similar samples (false negatives) that get wrongly pushed apart, degrading representations; existing detection is either batch-local or requires costly dataset-wide clustering.

Approach

Learn a per-anchor threshold via SGD on a convex quantile objective so that the top-alpha fraction of similar negatives across the entire dataset is identified as false negatives; per-iteration cost depends only on batch size, and the detected false negatives are filtered from the contrastive loss.

Key results

• On ImageNet100 with SogCLR (batch 128), GloFND improves semi-supervised linear evaluation average from 61.66 to 63.36 and reaches 53.10 F1 for false negative identification vs. 36.42 for FNC.
• On CC3M bimodal pretraining, SogCLR + GloFND improves Datacomp average from 24.87 to 25.37, and FastCLIP + GloFND reaches 25.37 vs. 24.76 for FastCLIP.
• Transfer learning across 8 datasets improves average accuracy from 62.67 (SogCLR) to 64.31 with GloFND, with per-epoch time rising only 2% (427 s to 435 s).

Measured effects reported in the paper

Surprising findings

• The mini-batch FNC baseline underperforms the SogCLR baseline in the bimodal setting (Datacomp 24.55 vs. 24.87).
• Applying false negative filtering too early hurts; GloFND performs best when started at epoch 70 of 200 and declines with later starts.

Limitations

• Gains are modest in some settings (e.g., bimodal retrieval for FastCLIP is not improved), and false negatives are removed by filtering only without exploring other treatments.
• Requires a warm-up period with a sufficiently trained encoder before starting, and the hyperparameter alpha must be tuned.
• Ground truth for false negative evaluation uses class labels, which is an approximation of semantic similarity.

Experimental setup

Datasets: ImageNet100 CC3M CIFAR-10 CIFAR-100 Food-101 Stanford Cars DTD Oxford-IIIT Pets Caltech-101 Oxford 102 Flowers

Benchmarks: ImageNet100 semi-supervised linear evaluation Food-101 CIFAR-10 CIFAR-100 Stanford Cars DTD Oxford-IIIT Pets Caltech-101 Oxford 102 Flowers Datacomp (38 tasks) ImageNet and distribution shifts Flickr30K

Metrics: top-1 accuracy precision recall F1-score Datacomp average p-value (paired t-test)

Baselines: SogCLR FNC FastCLIP SimCLR

Builds on: SogCLR FNC FastCLIP SimCLR MoCo

Closest papers by meaning

• Incremental False Negative Detection for Contrastive Learning (ICLR 2022 · similarity 0.90)
• Difficulty-Based Sampling for Debiased Contrastive Representation Learning (CVPR 2023 · similarity 0.86)
• Non-negative Contrastive Learning (ICLR 2024 · similarity 0.85)
• Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.84)
• Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling (AAAI 2025 · similarity 0.84)
• On the Similarities of Embeddings in Contrastive Learning (ICML 2025 · similarity 0.84)