Research map / Label-efficient and robust learning / Self-supervised and contrastive

An Augmentation-Aware Theory for Self-Supervised Contrastive Learning

ICML 2025 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Inductive bias and minimal representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an augmentation-aware error bound for InfoNCE-based self-supervised contrastive learning that links supervised downstream risk to contrastive risk plus same-class and same-image augmentation distances, then analyzes random crop and color distortion under a semantic label assumption.

Problem

Existing theory of contrastive learning either assumes conditionally independent positives (CURL) or abstract augmentation graphs, and does not explicitly explain how specific data augmentations affect downstream classification error.

Approach

Derives a decomposition of the InfoNCE risk over the number of negatives sharing the anchor's class, yielding an error bound with two augmentation-dependent distance terms (minimum same-class different-image distance and maximum same-image distance), whose trade-off with augmentation strength is analyzed for random resized crop and color distortion under a semantic label assumption.

Key results

• Pixel- and representation-level experiments show that as augmentation strength increases, the maximum same-image distance increases while the minimum same-class different-image distance decreases, on CIFAR-100 and TinyImageNet.
• The augmentation parameters minimizing the sum of the two distance terms also gave the highest linear probing accuracy on CIFAR-100 (SimCLR, ResNet-18, 1000 epochs).

Measured effects reported in the paper

Surprising findings

• Color distortion alone yields very poor linear probing accuracy (18.8 vs 56.3 with color+crop, as cited from prior work), suggesting crop is necessary for color to help.

Limitations

• Semantic label assumption and pixel-level generative model are idealized and serve only as an explanation rather than a generative model of images.
• Experiments are limited to CIFAR-100 and TinyImageNet with a single SimCLR setup and ResNet-18 backbone.
• Relies on a centered-representation assumption for the improved bound and on Lipschitz continuity for pixel-level bounds.

Experimental setup

Datasets: CIFAR-100 TinyImageNet

Benchmarks: CIFAR-100 TinyImageNet

Metrics: linear probing accuracy minimum same-class distance maximum same-image distance sum of distance terms

Baselines: CURL bound augmentation graph spectral bound augmentation overlap bound (sigma,delta)-measure bound

Builds on: CURL framework SimCLR spectral contrastive learning augmentation graph InfoNCE

Closest papers by meaning

• Towards the Generalization of Contrastive Self-Supervised Learning (ICLR 2023 · similarity 0.91)
• ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations (ICLR 2023 · similarity 0.91)
• Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.90)
• How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reduction (ICML 2025 · similarity 0.89)
• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression (ICLR 2024 · similarity 0.88)
• A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.88)