Research map / Label-efficient and robust learning / Recognition and long-tail
Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives a representation-based theory of weak-to-strong generalization (W2SG) in which the test-error gap (PredGap) is determined by principal kernels of weak and strong models, and proposes a label-free metric that predicts W2SG trends across molecular and NLP experiments.
Problem
Explaining when and why a strong model finetuned on weak-model labels outperforms its weak supervisor, including benign overfitting, without needing ground-truth labels to predict performance.
Approach
Characterize W2SG via kernels built from the principal components of weak and strong representations; the W2S error gap equals the projection of labels onto the subspace the strong model can learn from its principal representations but the weak model cannot, yielding a label-free metric ||P_s(I-P_w)|| that correlates with W2SG error.
Key results
- In the molecular setup (150 weak transformers, MolBERT strong), the label-free metric ||P_s(I-P_w)|| strongly correlates with W2S error and is a finer indicator than model size.
- Across the three setups (embedding models and end-to-end finetuned Qwen-7B across 52 LLMs total), high Spearman correlations are reported between W2S error and the metric.
- Theory shows benign overfitting occurs in a toy example: with eta=0.6, weak error ~0.4 while W2S error ~0.16 despite near-perfect training fit.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the strong model's principal representations to supervise on weak-model outputs | W2S model · Toy Gaussian example | test error: 0.4 -> 0.16 | Helped |
| Replicate weak-model errors aligned with the strong model's principal kernel | W2S model · Toy Gaussian example | test error | Hurt |
| Overfit weak-model errors orthogonal to the strong model's principal kernel | W2S model · Toy Gaussian example | test error | Helped |
| Use the representation-based overlap metric to predict W2S error | W2SG across molecular and NLP tasks · Eight datasets | classification error or MSE | Mixed |
Surprising findings
- Smaller weak models can outperform larger ones in W2S error, so model size is a poor predictor while the representation metric captures the trend.
- Errors aligned with the strong model's isotropic non-principal kernel are overfit yet do not harm test performance (benign overfitting).
Limitations
- Theory assumes linear heads on fixed representations and a high-probability decomposability condition; the full finetuning setting is only heuristically handled.
- Representations for end-to-end LLM finetuning are a heuristic (normalized activation maps), presented as proof of concept.
- Metric requires hyperparameters (alpha, beta) tuned per setup.
Experimental setup
Datasets: GuacaMol MoleculeNet (ESOL, FreeSolv, Lipop) via ChemBench ETHICS (Justice, Commonsense) SciQ Amazon Polarity Cosmos QA
Benchmarks: ESOL FreeSolv Lipop ETHICS Justice ETHICS Commonsense SciQ Amazon Polarity Cosmos QA
Metrics: W2S test error Spearman correlation between error and operator-norm metric
Baselines: Burns et al. W2SG disagreement theory model size as predictor
Builds on: weak-to-strong generalization benign overfitting NTK regime
Closest papers by meaning
- Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (ICML 2024 · similarity 0.93)
- The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge (NeurIPS 2026 · similarity 0.93)
- Quantifying the Gain in Weak-to-Strong Generalization (NeurIPS 2024 · similarity 0.92)
- Strong Empowered and Aligned Weak Mastered Annotation for Weak-to-Strong Generalization (AAAI 2025 · similarity 0.91)
- Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization (ICLR 2025 · similarity 0.91)
- Debate Helps Weak-to-Strong Generalization (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.