Research map / Multi-agent systems and games / LLM agents

Controllable User Simulation

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formalizes controllable user simulation as causal inference, shows post-hoc trajectory-label SFT induces look-ahead bias and variance explosion under policy shift ("controllability collapse"), and proposes a-priori, step-wise dynamic, and policy-conditioned training mitigations evaluated on WildChat and ConvApparel-V2.

Problem

Controllable user simulators for evaluating conversational agents are typically trained by SFT on post-hoc trajectory-level labels, which couple the control to the behavior policy's actions, yielding biased simulations and unstable counterfactual evaluation under new agent policies.

Approach

Frames controllable simulation as a causal/off-policy evaluation problem; shows that future-informed trajectory labels create look-ahead bias and geometric variance growth; proposes conditioning only on pre-interaction or step-wise observable controls (dynamic state z_t) and optionally conditioning on the target agent's policy.

Key results

• Trajectory-conditioned SFT (Scenario) averages 7.37 turns versus 4.31 for human data, while Dynamic State SFT matches at 4.78 turns and 109.5 words/turn (human 113.3).
• On held-out agents, static agnostic SFT's variance ratio grows roughly 1.42x per turn (estimated eta ~0.42), while dynamic state agent-aware SFT stays near 1 (eta ~0.01).
• Prompted simulators achieve high persona match (0.96) but entropy collapses from 1.04 to 0.728 on Persona+Goal.

Surprising findings

• Best-of-N rejection sampling with up to N=512 candidates per conversation achieves near-zero adherence (embedding similarity 0.28), failing entirely.
• Cognitive-profile trajectory-conditioned SFT shows poor persona adherence (0.56) compared with the prompted simulator (0.90).

Limitations

• Direct policy conditioning assumes white-box access to the evaluation policy.
• Dynamic state formulation lacks explicit global constraint mechanism, so global adherence cannot be directly compared with trajectory-forced baselines.
• Evaluation relies on proxy SFT agents and an LLM judge (Gemini) for both annotation and scoring; the theoretical results are stated under idealized assumptions.

Experimental setup

Datasets: WildChat ConvApparel

Benchmarks: WildChat ConvApparel-V2

Metrics: turn count words per turn task unresolved rate iterative refinement rate persona match goal match embedding similarity Shannon entropy of adherence scores variance ratio of total user words

Baselines: Unconditioned SFT Prompted Simulator Rejection Sampling Trajectory-Conditioned SFT Cognitive Profile SFT Agent-Aware SFT

Builds on: off-policy evaluation return-conditioned offline RL trajectory-conditioned user simulation

Closest papers by meaning

• Autoregressive Diffusion World Models for Off-Policy Evaluation of LLM Agents (NeurIPS 2026 · similarity 0.86)
• OdysSim: Building Foundation Models for Human Behavior Simulation (NeurIPS 2026 · similarity 0.86)
• $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment (ICML 2026 · similarity 0.86)
• Multimodal Policy Internalization for Conversational Agents (ICLR 2026 · similarity 0.85)
• Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
• Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues (AAAI 2025 · similarity 0.85)