Research map / Label-efficient and robust learning / Self-supervised and contrastive
Learning by Sorting: Self-supervised Learning with Group Ordering Constraints
Nina Shvetsova, Felix Petersen, Anna Kukleva, Bernt Schiele, Hilde Kuehne
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace InfoNCE with GroCo loss | SimCLR (ResNet50, 100 ep) · ImageNet k-NN k=20 | accuracy: 51.9 -> 60.5 | Improved |
| replace InfoNCE with GroCo loss | SimCLR (ResNet50, 100 ep) · ImageNet linear probing | Top-1 accuracy: 65.7 -> 69.2 | Improved |
| replace triplet loss with GroCo (ablation) | ResNet50 100 ep · ImageNet k-NN k=20 | accuracy: 53.8 -> 60.5 | Improved |
| pre-order positives and negatives before the sorting network (ablation) | GroCo · ImageNet k-NN k=20 | accuracy: 59.5 -> 60.5 | Improved |
| increase negatives from 10 to 20 at beta=1 (ablation) | GroCo · ImageNet k-NN k=20 | accuracy: 60.5 -> 54.6 | Hurt |
| increase batch size from 256 to 1024 (ablation) | GroCo · ImageNet k-NN k=20 | accuracy: 57.2 -> 60.5 | Improved |
| sample batches with many false negatives (ablation) | GroCo · ImageNet k-NN k=20 | accuracy: 60.5 -> 56.9 | Hurt |
| increase number of positive views from 2 to 4 (ablation) | GroCo · ImageNet k-NN k=20 | accuracy: 60.5 -> 61.8 | Improved |
| increase number of positive views from 2 to 4 (ablation) | GroCo · ImageNet linear probing | Top-1 accuracy: 69.2 -> 69.6 | No clear effect |
| use top-10 negatives without stop gradient (ablation) | GroCo and InfoNCE · ImageNet 100 epochs | training stability | Hurt |
| GroCo vs SwAV with multi-crop augmentation | SwAV · ImageNet linear probing 400 ep | Top-1 accuracy: 74.6 -> 73.7 | Hurt |
| plug GroCo loss into MoCo v3 with ViT-Small | MoCo v3 · ImageNet k-NN k=20 | accuracy: 66.8 -> 67.5 | Improved |
Closest papers by meaning
- Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision (ICML 2021 · similarity 0.86)
- Solving Inefficiency of Self-Supervised Representation Learning (ICCV 2021 · similarity 0.85)
- ISD: Self-Supervised Learning by Iterative Similarity Distillation (ICCV 2021 · similarity 0.85)
- Divide and Contrast: Self-Supervised Learning From Uncurated Data (ICCV 2021 · similarity 0.85)
- Contrastive Learning Meets Homophily: Two Birds with One Stone (ICML 2023 · similarity 0.85)
- High-Order Contrastive Learning with Fine-grained Comparative Levels for Sparse Ordinal Tensor Completion (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
