Research map / Large language models / Agents, code and math

Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation

NeurIPS 2026 · Large language models › Agents, code and math · approach: Long-horizon LLM agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DriftBench, a benchmark of 38 research briefs showing that under iterative pressure LLMs increase design complexity and often violate hard constraints they can still restate (knows-but-violates), with KBV ranging 8% to 99% across seven models.

Problem

Whether LLMs preserve fidelity to original research constraints during multi-turn iterative idea refinement, and whether constraint violations are explained by forgetting or recall failure.

Approach

Measures constraint adherence against structured briefs with binary-checkable hard constraints and banned moves, and introduces a restatement probe plus the knows-but-violates rate to separate declarative recall from behavioral adherence under single-shot, neutral, pressure, and checkpointed conditions.

Key results

• Knows-but-violates rate ranges from 8% (GPT-5.4) to 99% (Sonnet 4.6) under pressure, with four of seven models exceeding 50%.
• Constraint non-compliance rises from 35% (single-shot) to 54% (pressure); checkpointing partially recovers it to 47%.
• Judge shows near-perfect specificity (97%) but low sensitivity (15%) against human raters, so reported adherence scores are conservative.

Surprising findings

• Models accurately restate constraints they simultaneously violate, with near-perfect probe recall alongside violations, indicating the drift is not explained by simple forgetting.
• Temperature reduction increased KBV for Gemini Flash (76% to 83%), contrary to the expectation that lower temperature would reduce drift.

Limitations

• Pressure prompts may induce instruction arbitration between the latest user turn and the original brief, so drift may partly reflect adaptation.
• The 'Continue.' neutral control may itself act as weak pressure; no Anthropic frontier model (Opus 4.6) was run as a subject model.
• Single task domain (research ideation), long-horizon and open-ended; generalization to other tasks is untested.

Experimental setup

Datasets: 38 structured research briefs (24 scientific domains)

Benchmarks: DriftBench

Metrics: constraint adherence knows-but-violates rate fidelity alternative coverage complexity inflation surface fidelity gap restatement probe accuracy Cohen's kappa specificity sensitivity

Baselines: GPT-5.4-mini Gemini 3.1 Flash-Lite Llama-3.3-70B-Instruct Qwen3-235B

Builds on: MT-Bench MultiChallenge self-refine

Closest papers by meaning

• Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (ICLR 2024 · similarity 0.87)
• LIFEBENCH: Evaluating Length Instruction Following in Large Language Models (NeurIPS 2025 · similarity 0.87)
• $\texttt{ConflictBank}$: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMs (NeurIPS 2024 · similarity 0.87)
• The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.87)
• Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 · similarity 0.86)
• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.86)