Research map / Large language models / Agents, code and math
OmniSimulator: Aligning Small Language Models for Authentic Heterogeneous Behavior Modeling
Jiawei Chen, Ruoxi Xu, Boxi Cao, Zhuoqun Li, Ruotong Pan, Zhang yunfei, Zhirui Yang, Jingyan Chen, Tingting Gao, Han Li, Yaojie Lu, Xianpei Han and 3 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OmniSimulator, a method for aligning small language models to act as user simulators that model heterogeneous user behavior more authentically than standard chain-of-thought.
Problem
LLM-based user simulation for interactive systems is limited by the model's reasoning paradigm; standard chain-of-thought tends to degenerate into shallow patterns, capping simulation fidelity.
Approach
Aligns small language models for heterogeneous user behavior modeling, moving beyond standard chain-of-thought reasoning to improve authenticity of simulated users; specific mechanism not stated in the abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable native thinking mode | Qwen3-8B zero-shot · OmniBehavior | average score: 30.82 -> 30.77 | No clear effect |
| enable native thinking mode | Qwen3-235B zero-shot · OmniBehavior | average score: 33.15 -> 33.79 | No clear effect |
| add SFT on free-form teacher CoT | Qwen3-8B · OmniBehavior | average score: 30.82 -> 34.34 | Improved |
| use structured five-dimension CoT with rejection-sampled distillation instead of free CoT (ablation) | Qwen3-8B · OmniBehavior | average score: 34.34 -> 41.40 | Improved |
| add rejection sampling to teacher traces (ablation) | OmniSimulator Qwen3-8B · OmniBehavior | average score: 37.44 -> 41.40 | Improved |
| train on answers only instead of reasoning traces (ablation) | Qwen3-8B · OmniBehavior | average score: 30.82 -> 31.62 | No clear effect |
| use generic synthetic rationales instead of the structured schema (ablation) | Qwen3-8B · OmniBehavior | average score: 41.40 -> 37.85 | Hurt |
| include Historical Behavioral State dimension (ablation) | OmniSimulator Qwen3-1.7B · OmniBehavior binary | F1: 19.65 -> 28.80 | Improved |
| include User Profile Grounding dimension (ablation) | OmniSimulator Qwen3-1.7B · OmniBehavior textual | LLM-judge score: 31.99 -> 40.98 | Improved |
| include Decision Mode Identification dimension (ablation) | OmniSimulator Qwen3-1.7B · OmniBehavior binary | F1: 23.42 -> 28.80 | Improved |
| include Content Relevance Alignment dimension (ablation) | OmniSimulator Qwen3-1.7B · OmniBehavior continuous | 100 x (1-NMAE): 60.21 -> 62.12 | Improved |
| include all five schema dimensions versus none (ablation) | OmniSimulator Qwen3-1.7B · OmniBehavior binary | F1: 21.78 -> 28.80 | Improved |
Experimental setup
Baselines: standard chain-of-thought
Builds on: chain-of-thought
Closest papers by meaning
- OdysSim: Building Foundation Models for Human Behavior Simulation (NeurIPS 2026 · similarity 0.89)
- Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces (NeurIPS 2026 · similarity 0.89)
- Can LLMs Reliably Simulate Human Learner Actions? A Simulation Authoring Framework for Open-Ended Learning Environments (AAAI 2025 · similarity 0.88)
- Generating Computational Cognitive models using Large Language Models (NeurIPS 2025 · similarity 0.86)
- CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.86)
- Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviors (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
