Research map / Label-efficient and robust learning / Self-supervised and contrastive
Fast-MoCo: Boost Momentum-Based Contrastive Learning with Combinatorial Patches
Yuanzheng Ci, Chen Lin, Lei Bai, Wanli Ouyang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add combinatorial patches (divide, encode, combine) (ablation) | MoCo v3 · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 70.3 -> 73.5 | Improved |
| combine patch embeddings after encoding instead of combining images before encoding (ablation) | Fast-MoCo · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 71.8 -> 73.5 | Improved |
| divide image into non-overlapping patches instead of random sampling (ablation) | Fast-MoCo · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 72.9 -> 73.5 | Improved |
| use 6 combined pairs per image instead of 2 (ablation) | Fast-MoCo · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 72.6 -> 73.5 (4 pairs: 73.3) | Improved |
| average patch embeddings instead of max (ablation) | Fast-MoCo · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 64.6 -> 73.5 | Improved |
| use a different augmented view for the target (ablation) | Fast-MoCo · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 48.5 -> 73.5 | Improved |
| add multi-crop (ablation) | Fast-MoCo · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 73.5 -> 74.2 | Improved |
| add combinatorial patches | NNCLR · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 69.4 -> 72.5 | Improved |
| add combinatorial patches | BYOL · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 66.5 -> 73.6 | Improved |
| add combinatorial patches | SimSiam · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 68.1 -> 71.7 | Improved |
| add combinatorial patches with AdamW optimizer | MoCo v3 · ImageNet-1k linear eval (100 epochs) | top-1 accuracy: 69.3 -> 71.7 | Improved |
| train longer | Fast-MoCo · ImageNet-1k linear eval | top-1 accuracy: 73.5 -> 75.1 -> 75.5 (100, 200, 400 epochs) | Improved |
Closest papers by meaning
- Momentum Contrast for Unsupervised Visual Representation Learning (CVPR 2020 · similarity 0.89)
- Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning (NeurIPS 2021 · similarity 0.88)
- Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo (CVPR 2022 · similarity 0.87)
- Decoupled Contrastive Learning (ECCV 2022 · similarity 0.87)
- Self-Contrastive Learning: Single-Viewed Supervised Contrastive Framework Using Sub-network (AAAI 2023 · similarity 0.86)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
