Research map / Label-efficient and robust learning / Recognition and long-tail
Weak-to-Strong Generalization under Distribution Shifts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAVEN, a weak-to-strong generalization framework that jointly learns ensemble weights over multiple weak models and the strong model's parameters, with easy-sample guided initialization, to stay robust under distribution shift.
Problem
Weak-to-strong generalization breaks down under distribution shift between the weak supervisor's training data and the strong model's fine-tuning data, where the strong model can end up worse than the weak supervisors.
Approach
Treat the weak models as an ensemble of noisy annotators and learn adaptive, trainable combination weights jointly with the strong model's classifier (alternating updates), warm-started on samples where all weak models agree, so the strong model learns to up-weight the most reliable weak model for the target distribution.
Key results
- On OOD image classification, RAVEN(>3) reaches 19.27% average PGR versus 12.47% for Co-sup and 10.39% for Ens; RAVEN(3) gets 16.09% average PGR.
- On OOD text classification, RAVEN(3) achieves PGR of 74.56 on Llama-3.2-1B to Llama-3.1-8B and 85.88 on Qwen2.5-0.5B to Qwen2.5-7B, where naive W2S reaches 10.88 and 30.35.
- On preference alignment, RAVEN(3) reaches 58.94 PGR on Helpfulness-to-Harmlessness OOD versus 40.64 for Ens(3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RAVEN adaptive weighting instead of the strongest alternative ensemble baseline | Image classification weak-to-strong generalization · iWildCam, Camelyon17, and fMoW OOD | PGR: 10.90 -> 22.14; 17.13 -> 21.48; 13.24 -> 14.06 | Helped |
| Use RAVEN adaptive weighting instead of the strongest alternative baseline | Text classification weak-to-strong generalization · Amazon-Wilds OOD and InD; MedMCQA → MedQA OOD | PGR: 44.77 -> 74.56; 32.61 -> 85.88; 49.80 -> 53.60; 30.50 -> 42.60; 17.50 -> 19.00; 37.26 -> 44.00; 52.59 -> 71.95 | Helped |
| Use RAVEN DPO-R instead of the strongest alternative baseline | Qwen2.5-7B preference alignment · Helpfulness → Harmlessness OOD; Summarization → Human-like OOD; Harmlessness → Harmlessness InD | PGR: 40.64 -> 58.94; 41.88 -> 50.47; 42.04 -> 52.79 | Helped |
| Use Co-sup instead of RAVEN | DINO ViT-B/8 weak-to-strong generalization · ImageNet InD | PGR: 69.54 -> 68.60 | Hurt |
| Add ensemble supervision rather than using a single weak model (ablation) | Image classification weak-to-strong generalization · OOD, average across datasets | PGR: 1.80 -> 10.39 | Helped |
| Add easy-sample guided initialization (ablation) | RAVEN with ensemble supervision · Image classification OOD, average across datasets | PGR: 10.39 -> 10.67 | Helped |
| Add adaptive weighting (ablation) | RAVEN with ensemble supervision · Image classification OOD, average across datasets | PGR: 10.39 -> 14.28 | Helped |
| Add easy-sample guided initialization to ensemble plus adaptive weighting (ablation) | RAVEN · Image classification OOD, average across datasets | PGR: 14.28 -> 16.09 | Helped |
| Use the full RAVEN sample and weight schedule rather than adaptive weighting from the start (ablation) | RAVEN · Image classification OOD, average across datasets | PGR: 12.85 -> 16.09 | Helped |
Surprising findings
- Naive weak-to-strong generalization under distribution shift can yield negative PGR, with the strong model underperforming the weak supervisor (e.g., Meditron and Bayes baselines).
- The learned ensemble weights tend to place the highest weight on the best-performing weak model on the target data without access to target GT labels.
Limitations
- Weak models differ only by random seed in the main setting, which may not capture the diversity of real human annotators.
- Model-wise adaptive weights outperform the (model, sample)-wise variant, and the optimization challenges of the latter remain open.
- Evaluation relies on GPT-4o as judge for preference alignment and on specific benchmarks with synthetic or simulated weak supervision.
Experimental setup
Datasets: iWildCam Camelyon17 fMoW ImageNet ImageNet-C Amazon-Wilds MedMCQA MedQA HH-RLHF OpenAI Summarize from Feedback Human-Like DPO
Benchmarks: iWildCam Camelyon17 fMoW ImageNet Amazon-Wilds MedQA MedMCQA HH-RLHF OpenAI Summarize from Feedback Human-Like DPO
Metrics: accuracy performance gap recovered (PGR) GPT-4o win rate
Baselines: Naive weak-to-strong Auxiliary confidence loss Bootstrapping Vision superalignment Ensemble (uniform weights) Bayesian weakS-to-strong Co-supervised learning
Builds on: weak-to-strong generalization DPO Bayesian weakS-to-strong co-supervised learning
Closest papers by meaning
- Strong Empowered and Aligned Weak Mastered Annotation for Weak-to-Strong Generalization (AAAI 2025 · similarity 0.91)
- Debate Helps Weak-to-Strong Generalization (AAAI 2025 · similarity 0.91)
- Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (ICML 2024 · similarity 0.90)
- The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge (NeurIPS 2026 · similarity 0.90)
- Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions (ICML 2025 · similarity 0.90)
- From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.