Research map / Label-efficient and robust learning / Self-supervised and contrastive

Beyond Random Augmentations: Pretraining with Hard Views

ICLR 2025 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Data-efficient and robust self-supervision · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hard View Pretraining (HVP), a learning-free method that samples several augmented views per image, forwards them, and trains on the pair with the highest SSL loss, improving SimSiam, DINO, iBOT and SimCLR on ImageNet-1k.

Problem

Standard SSL pretraining relies on randomly sampled augmentations, which may be too easy to drive learning; prior hard-view methods either add auxiliary networks, need hyperparameter tuning, or were not validated at ImageNet-1k scale or with ViTs.

Approach

Sample N augmented views per image, compute the loss for all pairs using the current model state, and backpropagate only through the highest-loss pair (adversarial hard view selection), requiring no extra models or hyperparameter tuning.

Key results

• HVP sets a new state of the art for DINO ViT-B/16 with 78.8% linear accuracy (400 epochs) vs. 78.2% reported baseline, and 76.95% k-NN vs. 76.1%.
• HVP improves linear evaluation on ImageNet by about 1% on average for 100- and 300-epoch pretraining (e.g., DINO ViT-S/16 73.52 to 74.67 at 100 epochs; SimCLR RN50 63.37 to 65.33).
• HVP improves transfer, e.g., SimSiam RN50 Flowers102 linear accuracy 34.27 to 38.37 and COCO AP50 for DINO ViT-S/16 +0.47 at 100 epochs.

Measured effects reported in the paper

Surprising findings

• Selecting the pair with the best loss (cooperative) causes model collapse, with 0.1% linear accuracy.
• A manually scripted IoU-based augmentation policy that mimics the selection mostly degrades or does not improve performance, and a learned STN adversary underperforms the baseline.

Limitations

• Roughly 2x training-time overhead from extra forward passes for view selection.
• Pair selection can destabilize learning if the adversary is too strong (more views hurt performance).
• Results on most models are from 100/300-epoch runs with reduced budgets and limited to ImageNet-scale experiments.

Experimental setup

Datasets: ImageNet-1k CIFAR10 CIFAR100 Flowers102 iNat 21 Food101 COCO

Benchmarks: ImageNet-1k CIFAR10 CIFAR100 Flowers102 iNat 21 Food101 COCO

Metrics: top-1 linear accuracy k-NN accuracy fine-tuning accuracy AP AP50 AP75 IoU

Baselines: DINO SimSiam iBOT SimCLR random view sampling

Builds on: SimSiam DINO iBOT SimCLR

Closest papers by meaning

• Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.90)
• SEPT: Towards Scalable and Efficient Visual Pre-training (AAAI 2023 · similarity 0.88)
• One View Is Enough: In-the-Wild Monocular Pretraining for Novel View Generation (NeurIPS 2026 · similarity 0.88)
• Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.87)
• Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining (NeurIPS 2026 · similarity 0.87)
• Can Generative Models Improve Self-Supervised Representation Learning? (AAAI 2025 · similarity 0.87)