AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives

Chloe Li, Mary Phuong, Daniel Tan

ICLR 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning for reasoning data · 12 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add self-report fine-tuning (SRFT) to the modelGPT-4.1 interrogation · Hidden objective detection (4 stealth tasks)F1 score: 0 -> 0.98Improved
add SRFT to the modelGPT-4.1 interrogation · Hidden objective elicitation% details elicited: 0% -> 28-100%Improved
train on turn-two admissions only, not on generating false answers on-policy (ablation)SRFT · Hidden objective detection and elicitationdetection F1 and % details elicited: zero elicitation on all tasksHurt
add a system prompt telling the model to give false answers in turn-two-only training (ablation)SRFT (turn2-only) · Hidden objective detection and elicitationauditing performanceImproved
align admissions with ground truth rather than model belief on HLE hard questions (ablation)SRFT · Hidden objective detection and elicitationauditing performanceHurt
align admissions with model belief rather than ground truth (ablation)SRFT · Hidden objective detection and elicitationauditing performanceImproved
add 10% random flipping of turn-two admissions to training data (ablation)SRFT · Hidden objective detection and elicitationauditing performanceHurt
mix SRFT data 50/50 with general instruction tuning data (ablation)SRFT · Hidden objective detection and elicitationauditing performance: 50/50 token-balanced mixtureImproved
add a decoy objective to the agent during interrogationSRFT · Hidden objective elicitation (4 tasks)% details elicited: unchanged on 3 out of 4 tasksNo clear effect
add a decoy objective to the agent during interrogationPrefilled response attack on Claude Sonnet 4 · Hidden objective elicitation% details elicited: drops by 40-48% in 3 of 4 tasksHurt
apply SRFT to the modelGPT-4.1 · Instruction hierarchy compliancesystem instruction compliance rate (%): 72 -> 61Hurt
apply SRFT to the modelGPT-4.1 · GPQA-diamond, AIME 2025, HLEaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy