Research map / Label-efficient and robust learning / Semi-supervised learning
Deep Reference Priors: What is the best way to pretrain a model?
Yansong Gao, Rahul Ramesh, Pratik Chaudhari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a discrete reference prior with unlabeled target data and labeled source data instead of fine-tuning | CIFAR-100 transfer learning · Vehicles 2 → Vehicles 1 | accuracy: 66% -> 86% | Improved |
| Use source data and unlabeled target data for transfer without target labels | MNIST transfer learning · Target digits 5–9, 0 labeled target samples | accuracy: 86.6 -> 87.4 | Improved |
| Use EMA on particle weights (ablation) | Deep Reference Prior · CIFAR-10, 50 labeled samples | accuracy: 82.36 -> 85.45 | Improved |
| Vary prior order from 2 to 3, 4, or 5 (ablation) | Deep Reference Prior · CIFAR-10, 1000 labeled samples | accuracy: 91.76; 90.53; 91.51; 91.36 | No clear effect |
| Vary particle count from 4 to 2, 8, or 16 (ablation) | Deep Reference Prior · CIFAR-10, 1000 labeled samples | accuracy: 91.3; 91.76; 89.79; 90.72 | No clear effect |
| Remove the stop-gradient for weak-augmentation predictions (ablation) | Deep Reference Prior · CIFAR-10, 250 labeled samples | accuracy: 92.13 -> 10 | Hurt |
| Do not use Jensen's inequality to upper-bound the entropy (ablation) | Deep Reference Prior · CIFAR-10, 250 labeled samples | accuracy: 92.13 -> 86.55 | Hurt |
| Remove probability-threshold masking (ablation) | Deep Reference Prior · CIFAR-10, 250 labeled samples | accuracy: 92.13 -> 92.35 | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 0 of 1 on most shared results.
- FlexMatch — FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling (NeurIPS 2021) · better on 0 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors (NeurIPS 2022 · similarity 0.90)
- Inverse-Reference Priors for Fisher Regularization of Bayesian Neural Networks (AAAI 2023 · similarity 0.87)
- Learning to Select Pre-Trained Deep Representations With Bayesian Evidence Framework (CVPR 2016 · similarity 0.87)
- Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models (ICML 2024 · similarity 0.86)
- Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · similarity 0.86)
- Understanding the Transferability of Representations via Task-Relatedness (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
