Research map / Label-efficient and robust learning / Self-supervised and contrastive
Enhancing Representations Through Heterogeneous Self-Supervised Learning
Zhong-Yu Li, Bowen Yin, Yongxiang Liu, Li Liu, Ming‐Ming Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add heterogeneous auxiliary head (HSSL) | iBOT ViT-B/16 · ImageNet-1K linear probing (400 ep) | top-1: 77.8 -> 79.4 | Improved |
| add HSSL | DINO ViT-S/16 · ImageNet-1K k-NN (200 ep) | top-1: 61.2 -> 65.5 | Improved |
| add HSSL (600 ep) vs longer baseline (1600 ep) | iBOT ViT-B/16 · ImageNet-1K fine-tuning | top-1: 84.0 -> 84.1 | Improved |
| add HSSL | iBOT ViT-B/16 · ADE20K | mIoU: 47.9 -> 50.3 | Improved |
| use ConvNext auxiliary head instead of baseline (no head) (ablation) | ViT base · ImageNet-1K | top-1: 67.5 -> 72.7 | Improved |
| use ViT auxiliary head with ResNet base (similar architecture) (ablation) | ResNet base · ImageNet-1K | top-1: 63.2 -> 64.0 | Improved |
| add ConvNext head alongside ResMLP head (ablation) | ViT base · ImageNet-1K | top-1: 72.6 -> 73.7 | Improved |
| remove first shortcut in the auxiliary head (ablation) | ViT + ConvNext head · ImageNet-1K | top-1: 71.0 -> 72.7 | Improved |
| use auxiliary head to supervise base model (A->B) instead of base self-supervision (B->B) (ablation) | HSSL ViT · ImageNet-1K | top-1: 68.0 -> 72.7 | Improved |
| use MLP + token mixer head instead of MLP only (ablation) | HSSL ViT · ImageNet-1K | top-1: 70.0 -> 72.7 | Improved |
| use separate (non-shared) projection (ablation) | HSSL ViT · ImageNet-1K | top-1: 72.3 -> 72.7 | Improved |
| use serial instead of parallel connection of heads (ablation) | HSSL ViT · ImageNet-1K | top-1: 72.8 -> 73.9 | Improved |
Closest papers by meaning
- Patch-Level Representation Learning for Self-Supervised Vision Transformers (CVPR 2022 · similarity 0.88)
- Can Generative Models Improve Self-Supervised Representation Learning? (AAAI 2025 · similarity 0.87)
- A Unified Framework for Heterogeneous Semi-supervised Learning (CVPR 2025 · similarity 0.86)
- Probing the Mid-level Vision Capabilities of Self-Supervised Learning (CVPR 2025 · similarity 0.86)
- Self-Supervised Representation Learning with Meta Comprehensive Regularization (AAAI 2024 · similarity 0.86)
- Self-Supervised Representation Learning from Arbitrary Scenarios (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
