Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Towards Trustworthy Autonomous Systems via Conversations and Explanations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Doctoral consortium paper proposing CEMA, a causal explanation method for multi-agent autonomous systems based on counterfactual simulation and the Counterfactual Effect Size model, with plans to integrate it with language models and user mental models.
Problem
Black-box autonomous systems in multi-agent environments lack explanations that are causal, socially grounded, and accessible to lay users, which erodes trust, contestability, and human agency.
Approach
Extends the Counterfactual Effect Size Model of human causal selection to multi-agent systems: roll back the environment, sample counterfactual worlds from a learned probabilistic model, and rank features by correlation with the ego agent's queried actions to produce causal explanations without assuming a fixed causal structure.
Key results
- CEMA identifies correct and relevant causes in all four autonomous driving scenarios, even with many causally irrelevant agents present.
- A user study (N=200) found participants rank CEMA explanations at least as high as baseline human-elicited explanations.
- The user study indicates CEMA's explanations positively affect people's level of trust.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CEMA explanations instead of baseline explanations elicited from other human participants | CEMA explanations · CEMA user study (N=200) | participant explanation ranking | No clear effect |
| Provide CEMA explanations | CEMA · CEMA user study (N=200) | participant trust | Helped |
Limitations
- Current method only performs causal selection and does not generate fluent natural language or model user mental states.
- Evaluated only in autonomous driving so far.
- Initial results described as preliminary.
Experimental setup
Benchmarks: four autonomous driving scenarios with complex interactions
Metrics: correctness and relevance of identified causes participant rankings of explanations trust ratings
Baselines: explanations elicited from other human participants
Builds on: Counterfactual Effect Size Model Bayesian network causal model of AV decisions
Closest papers by meaning
- Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action (NeurIPS 2026 · similarity 0.86)
- Autonomous Policy Explanations for Effective Human-Machine Teaming (AAAI 2024 · similarity 0.86)
- ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence (NeurIPS 2026 · similarity 0.85)
- AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling (NeurIPS 2025 · similarity 0.85)
- Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents (NeurIPS 2024 · similarity 0.85)
- OpenDeception: Learning Deception and Trust in Human–AI Interaction via Multi-Agent Simulation (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.