Research map / Detection and segmentation / Open-set and few-shot detection
Your Self-Supervised Projection Head Captures Object Co-Occurrence Statistics
Arthur Aubret, Jochen Triesch, Céline Teulière
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Hypothesizes that common self-supervised learning pretext tasks implicitly model object co-occurrence statistics, and tests this using curated datasets of segmented objects from existing vision benchmarks.
Problem
Understanding the underlying principles that drive the success of self-supervised visual representation learning remains underexplored; the paper asks whether SSL pretext tasks implicitly capture object co-occurrence statistics.
Approach
Proposes that the projection head of SSL models encodes object co-occurrence statistics, testing this hypothesis with curated datasets of segmented objects derived from existing vision benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use an internal projection-head layer instead of the final backbone layer to encode object co-occurrences | pretrained SSL models (MoCoV3, BYOL, VICReg, DINO) · COCO-THINGS | RSA Kendall's tau: 9.0 -> 33.4 (DINO-ViT-B) | Improved |
| add a non-linear projection head during pre-training (ablation) | SimCLR, BYOL, DINO · COCO-THINGS co-occurrence encoding | RSA Kendall's tau: 2.3 -> 22.5 (BYOL) | Improved |
| add a non-linear projection head during pre-training (ablation) | SimCLR · ADE20k-THINGS | context accuracy: 24.5 -> 22.6 | Hurt |
| use projection head layer instead of backbone for human similarity judgments | pretrained SSL models · THINGS odd-one-out human judgments | RSA Kendall's tau: 18.3 -> 22.7 (BYOL-R50) | Improved |
| use a single linear projection layer instead of deeper head (ablation) | SimCLR, BYOL, DINO · COCO-THINGS | co-occurrence encoding vs backbone: SimCLR-1 mP.0 -0.4 vs backbone 4.3 | Hurt |
| sample positive pairs from different classes of the same context with probability p_c=0.3 (ablation) | SimCLR and DINO pre-training · ImageNet-1k, 100 epochs, n_c=300 | accuracy: up to +5.52% | Improved |
| raise p_c above 0.3 for cross-class same-context positive pairs (ablation) | SimCLR and DINO pre-training · ImageNet-1k, 100 epochs | accuracy | Hurt |
| increase the number of controlled co-occurrence contexts n_c (ablation) | SimCLR and DINO pre-training · ImageNet-1k, p_c=0.3 | accuracy | Improved |
| go deeper in the projection head for in-context one-shot category classification (ablation) | pretrained SSL models · COCO-THINGS | category classification accuracy: 92.9 -> 86.0 (DINO-ViT-B, B to P.3) | Hurt |
| use the last linear layer of the projection head (ablation) | pretrained SSL models · COCO-THINGS | RSA Kendall's tau: 13.9 vs 33.4 (DINO-ViT-B P.2 vs P.1 max) | Hurt |
| compute image-level instead of category-prototype similarities (ablation) | co-occurrence metrics · COCO-THINGS, ADE20k-THINGS, LVIS-LVIS | context accuracy | Hurt |
| vary number of context clusters (3 to 300) when scoring projection layers (ablation) | SimCLR, BYOL, DINO · LVIS-LVIS context accuracy | context accuracy | Mixed |
Experimental setup
Datasets: segmented objects curated from existing vision benchmarks
Closest papers by meaning
- Self-supervised visual learning from interactions with objects (ECCV 2024 · similarity 0.88)
- Object Discovery and Representation Networks (ECCV 2022 · similarity 0.88)
- Object Concepts Emerge from Motion (NeurIPS 2025 · similarity 0.86)
- Probing the Mid-level Vision Capabilities of Self-Supervised Learning (CVPR 2025 · similarity 0.86)
- Self-Organizing Visual Prototypes for Non-Parametric Representation Learning (ICML 2025 · similarity 0.86)
- Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
