Research map / Label-efficient and robust learning / Self-supervised and contrastive
OPERA: Omni-Supervised Representation Learning with Hierarchical Supervisions
Chengkun Wang, Wenzhao Zheng, Zheng Zhu, Jie Zhou, Jiwen Lu
This paper tried a common change: Train longer
increase pretraining epochs · ImageNet finetune and ADE20K · accuracy and mIoU: 78.7 at 50 PT epochs (finetune) Improved
What 387 accepted papers report for this change
- 211 improved
- 62 no clear effect
- 30 mixed
- 84 hurt
- Lightweight Federated Incremental Learning via Decoupled Replay ICML 2026 · increase local training epochs from 5 to 40 · CIFAR10 (α=1.0) · test accuracyImproved
- When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression ICML 2026 · train longer on the full path search space (overfitting stage) · SBM graph, 10 communities, pin=0.6 · accuracy: 1.0 -> 0.98Hurt
- Causes and Consequences of Representational Similarity in Machine Learning Models ICML 2026 · train longer on LLM continued pretraining to test insufficient adaptation · perplexity across overlap settings · perplexityImproved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add OPERA hierarchical supervision to MoCo-v3 (150 epochs vs MoCo-v3 300 epochs) | MoCo-v3 ResNet-50 · ImageNet linear probe | top-1 accuracy: 70.5 -> 73.7 | Improved |
| add OPERA to MoCo-v3 | MoCo-v3 ViT-B (BS 4096) · ImageNet finetune | top-1 accuracy: 83.0 -> 83.5 | Improved |
| add OPERA to MoCo-v3 | MoCo-v3 ViT-S · Oxford Flowers-102 transfer | top-1 accuracy: 90.3 -> 95.5 | Improved |
| add OPERA to MoCo-v3 on ADE20K segmentation | MoCo-v3 ViT-S · ADE20K | mIoU: 42.3 -> 43.6 | Improved |
| impose full supervision directly on backbone feature (arrangement A) instead of the OPERA proxy hierarchy (ablation) | OPERA · ImageNet / ADE20K ViT-S | classification vs segmentation | Mixed |
| increase pretraining epochs (ablation) | OPERA ViT-S · ImageNet finetune and ADE20K | accuracy and mIoU: 78.7 at 50 PT epochs (finetune) | Improved |
| increase number of MLP layers beyond 1 in the final block (ablation) | OPERA ViT-S · ImageNet | top-1 accuracy | Hurt |
| go from 1 to 2 MLP layers (ablation) | OPERA ViT-S · ADE20K | mIoU: +0.5 mIoU | Improved |
| enlarge hidden dimension of MLP (ablation) | OPERA ViT-S · ImageNet and ADE20K | accuracy and mIoU | No clear effect |
| use OPERA instead of supervised pretraining with MLP projector (ablation) | ViT-S supervised · ImageNet / ADE20K | top-1 accuracy / mIoU: 78.4 -> 78.8 top-1; 41.9 -> 42.4 mIoU | Improved |
| add MLP block to supervised pretraining (ablation) | Supervised ViT-S · ImageNet / ADE20K | top-1 accuracy / mIoU: 78.7 -> 78.4 top-1; 41.5 -> 41.9 mIoU | Mixed |
| add OPERA to MoCo-v3 at 150 epochs on COCO 1x | MoCo-v3 R50 · COCO 1x | AP box: 38.9 -> 38.9 | No clear effect |
Closest papers by meaning
- Free-Grained Hierarchical Visual Recognition (CVPR 2026 · similarity 0.86)
- Data Distillation: Towards Omni-Supervised Learning (CVPR 2018 · similarity 0.86)
- Exploring the Equivalence of Siamese Self-Supervised Learning via a Unified Gradient Framework (CVPR 2022 · similarity 0.86)
- Scaling and Benchmarking Self-Supervised Visual Representation Learning (ICCV 2019 · similarity 0.86)
- Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles (ICML 2023 · similarity 0.85)
- Learning Hierarchical Image Segmentation For Recognition and By Recognition (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
