Research map / Label-efficient and robust learning / Self-supervised and contrastive
Task-Customized Self-Supervised Pre-training with Scalable Dynamic Routing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SDR task-customized sub-nets instead of a unified full-ImageNet model | SimSiam · 11 downstream classification tasks (average) | linear-protocol accuracy (%): 72.90 -> 75.28 | Helped |
| use SDR task-customized sub-nets instead of a unified full-ImageNet model | BYOL · 11 downstream classification tasks (average) | linear-protocol accuracy (%): 75.95 -> 78.23 | Helped |
| use SDR task-customized sub-nets instead of a unified full-ImageNet model | SimSiam · PASCAL VOC test2007 | AP (%): 54.17 -> 55.65 | Helped |
| use SDR task-customized sub-nets instead of a unified full-ImageNet model | BYOL · PASCAL VOC test2007 | AP (%): 52.75 -> 53.53 | Helped |
| increase the number of sub-nets from 1 to 256 (ablation) | SDR · 11 downstream classification tasks (average) | kNN accuracy: 53.42 -> 58.13 | Helped |
| train sub-nets on separate semantic clusters rather than full ImageNet (ablation) | SDR · 11 downstream classification tasks | linear-protocol average accuracy (%): 72.50 -> 75.28 | Helped |
| use semantic clustering rather than random data splits (ablation) | SDR · 11 downstream classification tasks | linear-protocol average accuracy (%): 70.70 -> 75.28 | Helped |
| replace trained task-customized sub-nets with a pruned 22.6M SimSiam model (ablation) | SDR · 11 downstream classification tasks | linear-protocol average accuracy (%): 75.28 -> 56.78 | Hurt |
| replace SiamKD with L2 distillation (ablation) | SDR · 11 downstream classification tasks (average) | kNN accuracy: 58.13 -> 54.66 | Hurt |
| replace SiamKD with L2 distillation (ablation) | SDR · 11 downstream classification tasks | standard deviation of sub-net kNN accuracy: 2.38 -> 0.07 | Hurt |
Closest papers by meaning
- A Closer Look at Self-Supervised Lightweight Vision Transformers (ICML 2023 · similarity 0.88)
- DATA: Domain-Aware and Task-Aware Self-Supervised Learning (CVPR 2022 · similarity 0.88)
- Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.85)
- Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video (ICLR 2024 · similarity 0.85)
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.85)
- TADFormer: Task-Adaptive Dynamic TransFormer for Efficient Multi-Task Learning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.