Research map / Large language models / Language, safety and interpretability
Eliciting Language Model Behaviors with Investigator Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains Llama-3.1 8B "investigator" models with SFT, iterative DPO, and a Frank-Wolfe diversity objective to generate prompts that elicit target strings or rubric-specified behaviors (jailbreaks, hallucinations, aberrant persona) from target LMs.
Problem
Behavioral elicitation: finding diverse natural-language prompts that induce specific target behaviors (harmful outputs, hallucinations, aberrant personas) from language models, which is hard because the prompt space is combinatorially large and naive optimization yields mode-collapsed or gibberish
Approach
Amortize prompt search by training an investigator model to map target behaviors (strings or rubrics) to prompts, combining SFT on inverted forward data, DPO on elicitation reward, and an iterative Frank-Wolfe DPO variant whose regularizer penalizes previously discovered strategies, yielding diverse elicitation strategies that approximate posterior inference.
Key results
- Achieves 98.8% ASR on AdvBench Harmful Strings and 100% on the 50-task Harmful Behaviors subset against Llama-3.1 8B, versus 57.0% and 56% for GCG (numbers from prior work, not directly comparable).
- Full Frank-Wolfe reaches 85.2% ASR@25 on hallucination rubrics and 80.7% on aberrant behaviors, outperforming SFT (81.7% and 75.5%).
- Attacks transfer partially: 98% behavior ASR on Llama-3.3 70B-Turbo, 28% on GPT-4o (system prompt), and 26% on Claude 3.5 Sonnet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune with DPO after SFT | Llama-3.1 8B investigator · Pretraining strings test | Avg Elicit: -1.229 -> -0.922 | Helped |
| Aggregate the FW iterations rather than use DPO alone | Llama-3.1 8B investigator · Pretraining strings test | Entr: 0.400 -> 1.432 | Helped |
| Replace GCG with the trained investigator | Llama-3.1 8B · AdvBench Harmful Strings | ASR: 57.0% -> 98.8% | Helped |
| Replace GCG with the trained investigator | Llama-3.1 8B · AdvBench Harmful Behaviors (50-task subset) | ASR: 56% -> 100% | Helped |
| Use full Frank-Wolfe instead of SFT | Llama-3.1 8B-Instruct investigator · Hallucinations | ASR@25: 81.7% -> 85.2% | Helped |
| Use full Frank-Wolfe instead of SFT | Llama-3.1 8B-Instruct investigator · Aberrant Behaviors | ASR@25: 75.5% -> 80.7% | Helped |
| Use two-stage DPO rather than one-stage DPO (ablation) | Rubric elicitation investigator · In-distribution rubric elicitation | Verifier Score: -1.13 -> -0.70 | Helped |
| Use UltraChat instead of WildChat for SFT (ablation) | Llama-3.1 8B investigator · AdvBench Harmful Behaviors | ASR: 100% -> 100% | No clear effect |
| Use the FW-total aggregate rather than FW-1 alone (ablation) | Llama-3.1 8B investigator · AdvBench Harmful Strings | ASR: 98.8% -> 97.7% | Hurt |
| Use the FW-total aggregate rather than FW-1 alone (ablation) | Llama-3.1 8B investigator · AdvBench Harmful Strings | Entropy: 0.107 -> 3.416 | Helped |
| Use uniform averaging, eta_i = 1/i, instead of eta_i = 2/(i+1) (ablation) | Frank-Wolfe investigator · Pretraining string elicitation | Diversity score | Helped |
| Use uniform averaging, eta_i = 1/i, instead of eta_i = 2/(i+1) (ablation) | Frank-Wolfe investigator · Pretraining string elicitation | Elicitation score | Hurt |
Surprising findings
- Frank-Wolfe-style diversity regularization initially degrades elicitation log-probability at the second iteration before recovering, and DPO runs often collapse to repetition strategies.
- Small 1B Llama-3.2 investigators achieve comparable ASR to 8B investigators on AdvBench elicitation.
Limitations
- Reliance on LM verifiers and judges raises reward hacking concerns, observed in some results.
- Evaluation of harmful behaviors uses a GPT-4 autograder that is susceptible to false positives.
- Only single-turn elicitation is studied.
Experimental setup
Datasets: WildChat-1M-Full UltraChat FineWeb AdvBench TruthfulQA DSM-5 (parsed with GPT-4o)
Benchmarks: AdvBench Harmful Strings AdvBench Harmful Behaviors (50-task subset) TruthfulQA-derived hallucination rubrics DSM-5 aberrant behavior rubrics pretraining string elicitation (FineWeb gold pairs)
Metrics: attack success rate ASR@25 elicitation log-probability verifier score entropy best-of-k elicitation
Baselines: GCG Adaptive Attacks SFT DPO (FW-1) AutoDAN
Builds on: DPO GCG maximum entropy RL Frank-Wolfe optimization TRL
Closest papers by meaning
- CALM: Curiosity-Driven Auditing for Large Language Models (AAAI 2025 · similarity 0.87)
- Representation-Based Exploration for Language Models: From Test-Time to Post-Training (ICLR 2026 · similarity 0.87)
- CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.86)
- SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations (NeurIPS 2025 · similarity 0.86)
- Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models (NeurIPS 2026 · similarity 0.86)
- Eliciting Secret Knowledge from Language Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.