AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

ISD: Self-Supervised Learning by Iterative Similarity Distillation

Ajinkya Tejankar, Soroush Abbasi Koohpayegani, Vipin Pillai, Paolo Favaro, Hamed Pirsiavash

ICCV 2021 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Contrastive versus non-contrastive theory · 47 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use weak augmentation for the student instead of strong (teacher strong) (ablation)ISD · ImageNet NN, ResNet-50, 200 epNN accuracy: 58.0 -> 59.5Improved
use weak augmentation for the student instead of strong (teacher strong) (ablation)ISD · ImageNet 20-NN, ResNet-50, 200 ep20-NN accuracy: 61.2 -> 62.2Improved
increase memory bank from 128K to 512K (ablation)ISD · ImageNet NN, ResNet-50NN accuracy: 59.2 -> 59.0No clear effect
increase memory bank from 128K to 1024K (ablation)ISD · ImageNet NN, ResNet-50NN accuracy: 59.2 -> 59.5No clear effect
increase memory bank from 128K to 1024K (ablation)ISD · ImageNet 20-NN, ResNet-5020-NN accuracy: 62.0 -> 62.2No clear effect
set temperature to 0.02 versus 0.003 (ablation)ISD · ImageNet NN, ResNet-18, 130 epNN accuracy: 37.2 -> 39.7Improved
raise temperature from 0.02 to 0.06 (ablation)ISD · ImageNet NN, ResNet-18, 130 epNN accuracy: 39.7 -> 32.5Hurt
tune BYOL learning rate to 0.05 versus 0.01 (ablation)BYOL ResNet-18 · ImageNet NN, ResNet-18, 200 epNN accuracy: 37.3 -> 40.0Improved
replace MoCo hard contrastive labels with ISD soft similarity distillationMoCo · ImageNet linear, ResNet-18, 200 eptop-1 linear accuracy: 51.0 -> 53.8Improved
replace MoCo with ISD on unbalanced data, all 38 categoriesMoCo · Unbalanced ImageNet-38, dataset D3NN accuracy: 59.4 -> 61.9Improved
replace MoCo with ISD on unbalanced data, rare categories onlyMoCo · Unbalanced ImageNet-38, 30 rare categories, dataset D6NN accuracy: +3.8Improved
replace MoCo with ISD on unbalanced data, dataset D10 rare categoriesMoCo · Unbalanced ImageNet-38, 30 rare categories, D10NN accuracy: 59.8 -> 51.5Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy