Research map / Label-efficient and robust learning / Knowledge distillation
Temporal Knowledge Consistency for Unsupervised Visual Representation Learning
Weixin Feng, Yuanjiang Wang, Lihua Ma, Ye Yuan, Chi Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TKC temporal teachers | MoCo v2 · ImageNet linear eval (200 epochs) | top-1 accuracy: 67.5 -> 69.0 | Improved |
| add TKC temporal teachers | BYOL · ImageNet linear eval (100 epochs) | top-1 accuracy: 70.1 -> 72.4 | Improved |
| add TKC temporal teachers | MoCo v2 (AlexNet) · ImageNet linear eval conv4 | top-1 accuracy: 39.0 -> 46.2 | Improved |
| add TKC temporal teachers | MoCo v2 (AlexNet) · ImageNet linear eval conv5 | top-1 accuracy: 42.8 -> 44.0 | Improved |
| add one temporal teacher to EMA teacher (h=1 to h=2) (ablation) | TKC · ImageNet linear eval (AlexNet) | top-1 accuracy: 39.9 -> 42.2 | Improved |
| use three teachers instead of two (h=2 to h=3) (ablation) | TKC · ImageNet linear eval (AlexNet) | top-1 accuracy: 42.2 -> 43.5 | Improved |
| increase teachers beyond three (h=3 to h=6) (ablation) | TKC · ImageNet linear eval (AlexNet) | top-1 accuracy: 43.5 -> 41.8 | Hurt |
| use 4-layer MLP knowledge transformer instead of 2-layer (ablation) | TKC · ImageNet linear eval (100 epochs) | top-1 accuracy: 65.91 -> 66.21 | Improved |
| use bottleneck MLP knowledge transformer instead of 2-layer (ablation) | TKC · ImageNet linear eval (100 epochs) | top-1 accuracy: 65.91 -> 66.31 | Improved |
| add TKC with frozen backbone detection | MoCo v2 · PASCAL VOC detection (frozen backbone) | AP: 51.7 -> 52.7 | Improved |
| add TKC with frozen backbone detection | MoCo v2 · COCO detection (frozen backbone) | AP: 29.2 -> 34.7 | Improved |
| add TKC when fine-tuning on COCO detection | MoCo v2 · COCO detection (fine-tune) | AP: 40.1 -> 40.4 | No clear effect |
Closest papers by meaning
- Train a One-Million-Way Instance Classifier for Unsupervised Visual Representation Learning (AAAI 2021 · similarity 0.87)
- Towards Efficient and Effective Self-Supervised Learning of Visual Representations (ECCV 2022 · similarity 0.87)
- Unsupervised Feature Learning via Non-Parametric Instance Discrimination (CVPR 2018 · similarity 0.87)
- Learning from Memory: Non-Parametric Memory Augmented Self-Supervised Learning of Visual Features (ICML 2024 · similarity 0.86)
- Temporally Consistent Object-Centric Learning by Contrasting Slots (CVPR 2025 · similarity 0.86)
- Towards flexible perception with visual memory (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
