Research map / Large language models / Instruction and fine-tuning
Removing Sandbagging in LLMs by Training with Weak Supervision
Emil Ryd, Henning Bartsch, Julian Stastny, Joe Benton, Vivek Hebbar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SFT before RL with very weak supervisor (ablation) | RL on gpt-oss-120b sandbagger · Olympiads | fraction of benign performance elicited: 0.13 ± 0.00 -> 0.88 ± 0.03 | Improved |
| add SFT before RL with very weak supervisor (ablation) | RL on gpt-oss-120b sandbagger · Super GPQA | fraction of benign performance elicited: 0.30 ± 0.01 -> 0.99 ± 0.03 | Improved |
| add SFT before RL with weak supervisor (ablation) | RL on gpt-oss-120b sandbagger · Olympiads | fraction of benign performance elicited: 0.34 ± 0.18 -> 0.86 ± 0.03 | Improved |
| add RL after SFT with very weak supervisor (ablation) | SFT on gpt-oss-120b sandbagger · Olympiads | fraction of benign performance elicited: 0.40 ± 0.05 -> 0.88 ± 0.03 | Improved |
| add RL after SFT with weak supervisor (ablation) | SFT on gpt-oss-120b sandbagger · Olympiads, Super GPQA, Code Contests | fraction of benign performance elicited: 0.85 ± 0.06 -> 0.86 ± 0.03 (Olympiads) | No clear effect |
| SFT on very weak (Llama 3.1 8B) demonstrations instead of weak (GPT-4o-mini) (ablation) | SFT on gpt-oss-120b sandbagger · Olympiads | fraction of benign performance elicited: 0.85 ± 0.06 -> 0.40 ± 0.05 | Hurt |
| RL with ground truth on only 10 samples | gpt-oss-120b sandbagger · Olympiads, Super GPQA, Code Contests | elicitation | No clear effect |
| RL against weak or very weak supervisor alone | gpt-oss-120b sandbagger · Olympiads, Super GPQA | ground truth performance | Hurt |
| SFT on 10 ground truth samples | gpt-oss-120b sandbagger · Super GPQA | elicited performance: 99% on Super GPQA | Improved |
| make training distinguishable from deployment (situationally aware sandbagger), very weak supervisor (ablation) | SFT+RL · Olympiads | fraction of benign performance elicited: 88% -> 36% | Hurt |
| make training distinguishable from deployment (situationally aware sandbagger), very weak supervisor (ablation) | SFT+RL · Super GPQA | fraction of benign performance elicited: 99% -> 60% | Hurt |
| make training distinguishable from deployment (situationally aware sandbagger), very weak supervisor (ablation) | SFT+RL · Code Contests | fraction of benign performance elicited: 93% -> 66% | Hurt |
Closest papers by meaning
- Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models (NeurIPS 2025 · similarity 0.86)
- Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use (ICML 2026 · similarity 0.86)
- Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback (NeurIPS 2026 · similarity 0.86)
- AI Sandbagging: Language Models can Strategically Underperform on Evaluations (ICLR 2025 · similarity 0.85)
- Debate Helps Weak-to-Strong Generalization (AAAI 2025 · similarity 0.85)
- Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
