Research map / Label-efficient and robust learning / Self-supervised and contrastive
Momentum Contrast for Unsupervised Visual Representation Learning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a queue and momentum-updated key encoder instead of a memory bank (ablation) | MoCo contrastive pretraining · ImageNet linear classification | top-1 accuracy: 58.0% memory bank; 2.6% worse than MoCo | Helped |
| Use a momentum-updated key encoder rather than momentum 0.9 (ablation) | MoCo · ImageNet linear classification, K=4096 | accuracy: 55.2 -> 59.0 | Helped |
| Increase key-encoder momentum from 0.999 to 0.9999 (ablation) | MoCo · ImageNet linear classification, K=4096 | accuracy: 59.0 -> 58.9 | No clear effect |
| Remove the momentum update (ablation) | MoCo · ImageNet linear classification, K=4096 | training convergence | Hurt |
| Use MoCo dictionary mechanism rather than end-to-end batch dictionary (ablation) | Contrastive pretraining · VOC test2007, Faster R-CNN R50-C4 | AP: 54.6 -> 55.9 | Helped |
| Use MoCo dictionary mechanism rather than memory bank (ablation) | Contrastive pretraining · VOC test2007, Faster R-CNN R50-C4 | AP: 54.9 -> 55.9 | Helped |
| Pretrain on Instagram-1B rather than ImageNet-1M | MoCo · VOC test2007, Faster R-CNN R50-C4 | AP: 55.9 -> 57.2 | Helped |
| Pretrain on Instagram-1B rather than ImageNet-1M | MoCo · LVIS v0.5 instance segmentation | mask AP: 24.1 -> 24.9 | Helped |
| Pretrain on Instagram-1B rather than ImageNet-1M | MoCo · iNaturalist 2018 | accuracy: 65.6 -> 65.8 | No clear effect |
| Pretrain on Instagram-1B rather than ImageNet-1M | MoCo · COCO val2017, Mask R-CNN R50-FPN, 2× | bbox AP: 40.8 -> 41.1 | Helped |
| Pretrain with MoCo rather than ImageNet supervision | ResNet-50 features · VOC val2012 semantic segmentation | mIoU: 74.4 -> 73.6 | Hurt |
| Remove shuffling BN (ablation) | MoCo and end-to-end contrastive training · ImageNet pretext training and kNN validation | kNN classification accuracy | Hurt |
Closest papers by meaning
- Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations (NeurIPS 2021 · similarity 0.89)
- VideoMoCo: Contrastive Video Representation Learning With Temporally Adversarial Examples (CVPR 2021 · similarity 0.89)
- Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning (NeurIPS 2021 · similarity 0.88)
- CO2: Consistent Contrast for Unsupervised Visual Representation Learning (ICLR 2021 · similarity 0.88)
- Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning (CVPR 2023 · similarity 0.87)
- BECLR: Batch Enhanced Contrastive Few-Shot Learning (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.