Research map / Label-efficient and robust learning / Self-supervised and contrastive
Efficient Visual Pretraining With Contrastive Detection
Olivier J. Hénaff, Skanda Koppula, Jean-Baptiste Alayrac, Aaron van den Oord, Oriol Vinyals, João Carreira
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace global pooling with a single FH mask per image (ablation) | SimCLR · COCO pretrain, COCO frozen-feature | AP bbox: 31.6 -> 31.2 | Hurt |
| duplicate the global mask 16 times (ablation) | SimCLR · COCO pretrain, COCO frozen-feature | AP bbox: 31.6 -> 31.5 | No clear effect |
| sample 16 FH masks per image (dense mask sampling) (ablation) | SimCLR with FH masks · COCO pretrain, COCO frozen-feature | AP bbox: 31.2 -> 33.4 | Improved |
| replace SimCLR with full DetCon_S | SimCLR · COCO pretrain, COCO frozen-feature | AP bbox: 31.6 -> 33.4 | Improved |
| replace SimCLR with full DetCon_S | SimCLR · COCO pretrain, COCO frozen-feature | AP mask: 29.2 -> 30.6 | Improved |
| gather negatives from the entire batch (FH masks) (ablation) | DetCon_S · COCO frozen-feature, FH masks | AP bbox: 32.2 -> 33.6 | Improved |
| gather negatives from the entire batch (GT masks) (ablation) | DetCon_S · COCO frozen-feature, GT masks | AP bbox: 38.5 -> 37.0 | Hurt |
| make predictions across two augmented views (FH masks) (ablation) | DetCon_S · COCO frozen-feature, FH masks | AP bbox: 27.7 -> 32.2 | Improved |
| make predictions across two augmented views (GT masks) (ablation) | DetCon_S · COCO frozen-feature, GT masks | AP bbox: 38.8 -> 38.5 | Hurt |
| use DetCon_B instead of BYOL pretraining (300 epochs) | BYOL · PASCAL segmentation | mIoU: 74.7 -> 75.6 | Improved |
| use DetCon_B instead of BYOL pretraining (300 epochs) | BYOL · Cityscapes segmentation | mIoU: 73.4 -> 75.1 | Improved |
| use DetCon_B instead of BYOL (1x fine-tune) | BYOL · COCO detection | AP bbox: 41.6 -> 42.7 | Improved |
Closest papers by meaning
- Aligning Pretraining for Detection via Object-Level Contrastive Learning (NeurIPS 2021 · similarity 0.92)
- Boost Supervised Pretraining for Visual Transfer Learning: Implications of Self-Supervised Contrastive Representation Learning (AAAI 2022 · similarity 0.92)
- When Does Contrastive Visual Representation Learning Work? (CVPR 2022 · similarity 0.91)
- What Makes Instance Discrimination Good for Transfer Learning? (ICLR 2021 · similarity 0.90)
- A Broad Study on the Transferability of Visual Representations With Contrastive Learning (ICCV 2021 · similarity 0.89)
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
