Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MIMIC, which treats VLM-generated language descriptions as inner speech, learns a CVAE to generate it from observation history, and conditions a diffusion behavior cloner on it to get diverse, steerable imitation on D3IL and Overcooked.
Problem
Behavior cloning struggles to capture the diversity and non-Markovian nature of human behavior and offers no inference-time steering of behavior, which limits its use as a human surrogate for human-AI coordination.
Approach
Model human actions through a latent inner-speech mediator p(a|s)=∫p(a|s,m)p(m|s)dm: VLM-generated behavior descriptions (CLIP-encoded) scaffold a CVAE that generates inner speech online, and a classifier-free-guided diffusion policy conditions on it, so designers can steer behavior by supplying text.
Key results
- On D3IL Aligning, MIMIC-S reaches 0.8021 success rate vs 0.6645 for BC, and MIMIC-E reaches entropy 0.6148 vs 0.4743.
- On Overcooked Cramped room, MIMIC attains collective reward 151.8 vs 115.8 for BC with a greedy partner.
- Steering with text descriptions yields successful, controllable trajectories judged by GPT-4o as matching the requested behavior.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add inner-speech mediation with MIMIC-S instead of plain BC | DDPM-T behavior cloning · Aligning | success rate: 0.6645 -> 0.8021 | Helped |
| Add inner-speech mediation with MIMIC-E instead of plain BC | DDPM-T behavior cloning · Aligning | entropy: 0.4743 -> 0.6148 | Helped |
| Add inner-speech mediation with MIMIC-S instead of plain BC | DDPM-T behavior cloning · Sorting-Vision | success rate: 0.7972 -> 0.8417 | Helped |
| Add inner-speech mediation with MIMIC-S instead of plain BC | DDPM-T behavior cloning · Stacking, 2 boxes | success rate: 0.4879 -> 0.6074 | Helped |
| Add MIMIC inner speech | DDPM-T behavior cloning with a Greedy Overcooked partner · Cramped room | collective reward: 115.8 ± 3.86 -> 151.8 ± 2.45 | Helped |
| Remove inner speech during simulation (ablation) | MIMIC · Aligning | success rate | Hurt |
| Replace generated inner speech with K-means cluster representations (ablation) | MIMIC · Aligning | success rate | Hurt |
| Replace CLIP text embeddings with MPNET embeddings (ablation) | MIMIC-S · Aligning | success rate | Hurt |
| Replace CLIP text embeddings with MPNET embeddings (ablation) | MIMIC-E · Aligning | entropy | Hurt |
| Use o4-mini instead of GPT-4o for VLM scaffolding (ablation) | MIMIC · Aligning | success rate | Hurt |
| Use o4-mini instead of other VLMs for scaffolding (ablation) | MIMIC · Aligning | entropy | Helped |
| Use Qwen2.5-VL-72B-Instruct for VLM scaffolding (ablation) | MIMIC · Aligning | entropy | Hurt |
Surprising findings
- o4-mini descriptions produced the highest diversity (entropy) but the lowest success rate, which the authors attribute to CLIP failing to separate its nuanced behaviors.
- Random inner speech performed better than BC, while K-means cluster-based inner speech was the worst variant.
Limitations
- Inner speech quality depends on VLM-generated captions, so progress is gated by VLM capability.
- Polling window W needs task-specific calibration; too frequent updates cause instability and too infrequent updates reduce drift correction.
- Latent CVAE codes are not directly interpretable, and evaluation is limited to dyadic settings with a proxy human.
Experimental setup
Datasets: D3IL Overcooked
Benchmarks: D3IL Aligning D3IL Sorting D3IL Stacking Overcooked
Metrics: success rate behavioral entropy distance Wasserstein distance collective reward action Wasserstein distance
Baselines: BC (DDPM-T) Thought Cloning BESO BeT K-means inner speech random inner speech
Builds on: diffusion policy conditional variational autoencoder thought cloning classifier-free guidance behavior cloning
Closest papers by meaning
- Policy Improvement using Language Feedback Models (NeurIPS 2024 · similarity 0.88)
- Behavioral Exploration: Learning to Explore via In-Context Adaptation (ICML 2025 · similarity 0.88)
- Self-Improving Embodied Foundation Models (NeurIPS 2025 · similarity 0.87)
- Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies (NeurIPS 2025 · similarity 0.86)
- Imitating Language via Scalable Inverse Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Text-Aware Diffusion for Policy Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.