Research map / Large language models / Retrieval and knowledge
Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a tree-search framework that generates model-adaptive, answer-preserving distractions to stress-test LLM contextual robustness, and finds DPO mitigates the vulnerability better than prompt-based methods.
Problem
LLMs lose accuracy when given semantically coherent but task-irrelevant context, yet existing static fixed-template or retrieval-based distractions are weak against contemporary models, so robustness evaluation is insufficient.
Approach
A proxy LLM generates distraction context additions guided by a priority-queue tree search whose value function combines the victim's success rate (sampled n times) with a depth penalty; a classifier pre-filters distraction-susceptible questions, and early stopping and length/semantic constraints keep distractions efficient and answer-preserving.
Key results
- Adaptive distractions cause an average accuracy drop of about 52% (to 0.323 average accuracy) across seven models, versus 15.2% for DyVal2 in the comparison.
- Classifier filtering raises perturbation success from 59% to 82% and lowers cost per successful distraction by 21.9%.
- DPO raised Phi-3.5-mini accuracy under adaptive distraction from 0.195 to 0.680, while prompt-based methods gave only marginal gains (mostly under +0.03).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive distractions to model inputs | mainstream LLMs · CommonsenseQA, OpenbookQA, TruthfulQA, and MMLU | accuracy: 0.857 -> 0.220; 0.897 -> 0.228; 0.607 -> 0.160; 0.787 -> 0.255 | Hurt |
| add adaptive distractions to model inputs | Claude-3.5-Sonnet · CommonsenseQA, OpenbookQA, TruthfulQA, and MMLU | accuracy: 0.873 -> 0.345; 0.953 -> 0.529; 0.840 -> 0.734; 0.877 -> 0.645 | Hurt |
| use tree-search adaptive distraction instead of ICA, SPD, or DyVal2 | seven evaluated models · four-dataset average | accuracy: 0.843 -> 0.323 | Hurt |
| remove the depth penalty from the value function (ablation) | adaptive distraction search · four datasets | distraction success rate: 59% -> 57% | Hurt |
| remove the success-rate term from the value function (ablation) | adaptive distraction search · four datasets | distraction success rate: 59% -> 53% | Hurt |
| add classifier filtering of hard-to-distract questions (ablation) | adaptive distraction generation · four datasets | perturbation success rate: 59% -> 82% (+38.9%) | Helped |
| add classifier filtering of hard-to-distract questions (ablation) | adaptive distraction generation · four datasets | cost per successful distraction: 0.0082 -> 0.0064 (21.9%) | Helped |
| apply prompt-based mitigation | seven LLMs · adaptive distraction examples | accuracy: +0.026; +0.003; +0.018; -0.008; +0.009; +0.002; -0.013 | Mixed |
| apply few-shot CoT or self-consistency prompting | GPT-4o-mini, GPT-4o, Qwen2.5-72B, and Llama-3.1-8B · adaptive distraction examples | accuracy: 0.185 -> 0.238; 0.390 -> 0.455; 0.334 -> 0.352; 0.247 -> 0.285 | Helped |
| DPO fine-tune on preference data | Gemma-2-2B, Qwen2.5-7B, and Phi-3.5-mini · adaptive distraction examples | accuracy: 0.257 -> 0.432; 0.212 -> 0.440; 0.195 -> 0.680 | Helped |
| use SFT instead of the untrained model | Gemma-2-2B, Qwen2.5-7B, and Phi-3.5-mini · adaptive distraction examples | accuracy: 0.257 -> 0.305; 0.212 -> 0.278; 0.195 -> 0.261 | Helped |
| DPO fine-tune | Gemma-2-2B, Qwen2.5-7B, and Phi-3.5-mini · clean questions | accuracy: 0.450 -> 0.398; 0.480 -> 0.411; 0.720 -> 0.697 | Hurt |
Surprising findings
- Distractions generated by a model against itself were more adversarial than those generated by a stronger model (e.g., GPT-4o-mini scored 0.185 on its own distractions vs 0.235 on GPT-4o's).
- Static GSM-IC-style distractions cause only about 1-4% drops on advanced models, far below the adaptive method's drops.
Limitations
- Human evaluation covered only 800 GPT-4o-mini-enhanced questions judged by five undergraduates, and the paper relies on automatic semantic checks.
- Several claimed numbers are inconsistent across tables and the abstract's 45% figure differs from the 52% average in the comparison table.
- Mitigation via DPO was tested on only three small open models, and the DPO gains were not compared against proprietary models trained the same way.
Experimental setup
Datasets: MMLU CommonsenseQA OpenbookQA TruthfulQA MATH-500
Benchmarks: MMLU CommonsenseQA OpenbookQA TruthfulQA MATH-500
Metrics: accuracy accuracy drop F0.5 score distraction success rate cost per successful distraction semantic equivalence rate answer consistency rate
Baselines: Irrelevant Context Augmentation (ICA) Single-Prompt Distraction (SPD) DyVal2 GSM-IC Zero-shot CoT Few-shot CoT Few-shot ICL Self-Consistency
Builds on: Monte Carlo Tree Search Direct Preference Optimization DyVal2 GSM-IC
Closest papers by meaning
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.88)
- Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack (NeurIPS 2024 · similarity 0.88)
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.87)
- NExtLong: Toward Effective Long-Context Training without Long Documents (ICML 2025 · similarity 0.87)
- Leveraging Attention to Effectively Compress Prompts for Long-Context LLMs (AAAI 2025 · similarity 0.87)
- Let's (not) just put things in Context: Test-time Training for Long-context LLMs (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.