AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Exploration Hacking: Can LLMs Learn to Resist RL Training?

Eyon Jang, Damon Falck, Joschka Braun, Nathalie Maria Kirch, Achyutha Menon, Perusha Moodley, Scott Emmons, Roland S. Zimmermann, David Lindner

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Calibration and knowledge in LLM tuning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune (RL-lock) the model instead of only prompting it to underperform (ablation)Qwen3-14B · KernelBench RL elicitationresistance to elicitation: recovers within 100 steps -> suppressed for over 500 stepsImproved
add few-shot (3-shot) examples to prompt locking (ablation)Qwen3-14B · KernelBenchresistance to RL elicitationImproved
add more shots to prompt locking (ablation)Qwen3-14B · BigCodeBenchresistance to RL elicitationNo clear effect
use deterministic shortest-answer strategy instead of random wrong answer (ablation)RL-locked Qwen3-14B · WMDP-Bioaccuracy under RL elicitation: 25% -> over 50% (wrong answer) vs flat (shortest answer)Improved
increase locking strength (80% vs 95% reliability) (ablation)RL-locked Qwen3-14B · WMDP-Bioresistance to elicitationNo clear effect
use stronger SFT locking (180 vs 40 steps) (ablation)SFT-locked Qwen3-14B · BigCodeBench RL elicitationmalign performanceImproved
use continuous multi-test-case reward rather than multiple choiceRL elicitation · KernelBench, BigCodeBench vs WMDP-Biodifficulty of resisting elicitationHurt
add action-only or CoT monitor (gpt-5-mini)exploration hacking model organisms · BigCodeBenchdetection ROC-AUCImproved
add Gaussian weight noise to LoRA adaptersBigCodeBench model organism · BigCodeBench with triggeraccuracy: transient improvement for sigma <= 0.02, then collapseMixed
add SFT on benign traceslocked model organism · WMDP-Bio [eval] segmentaccuracy: 80.0% -> 98.0%Improved
add SFT on filtered Qwen3-4B traceslocked model organism · BigCodeBenchmalign pass@1Improved
increase scaffolding from minimal to full in system prompt (ablation)frontier models · WMDP/VCT auditexploration hacking rateImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy