Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Autonomous Policy Explanations for Effective Human-Machine Teaming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A doctoral thesis proposal summarizing work on autonomous policy explanations (semantic, AR-based visual, and counterfactual justifications) to improve human-robot teaming, trust, and coaching.
Problem
Autonomous agents often fail to communicate their decision-making rationale, capabilities, and limitations to human teammates, which hurts shared understanding, trust, and value alignment in human-machine teams.
Approach
Combines multiple explanation modalities for robots: a RARE coaching framework with semantic explanations from planning predicates solved via integer programming, AR-based prescriptive/descriptive visual guidance (MARS), and value-of-information-timed policy/environment justifications, with a proposed future integration with foundation models.
Key results
- In the RARE sudoku study, irreversible mistakes dropped to 20% with justifications versus 80% without.
- The semantic explanation optimizer substantially outperforms Hayes and Shah 2017 and is now practical for online use.
- Policy-based justifications promote higher compliance and quicker decision-making, while environment-based justifications improve perceived interpretability, intelligence, and trustworthiness.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add robot justifications for corrective actions | RARE · Collaborative color-based sudoku game | irreversible mistakes: 80% -> 20% | Helped |
| Combine prescriptive and descriptive visual guidance | MARS · 3D AR-based human-robot Minesweeper game | trust | Helped |
| Combine prescriptive and descriptive visual guidance | MARS · 3D AR-based human-robot Minesweeper game | performance | Helped |
| Use policy-based rather than environment-based justifications | robot recommendations · online human-subjects study | compliance | Helped |
| Use policy-based rather than environment-based justifications | robot recommendations · online human-subjects study | decision-making speed | Helped |
| Use environment-based rather than policy-based justifications | robot recommendations · online human-subjects study | interpretability | Helped |
| Use environment-based rather than policy-based justifications | robot recommendations · online human-subjects study | trustworthiness | Helped |
| Use strategic timing rather than constant or timed-interval timing for justifications | robot recommendations · expert-feedback study | perceived usefulness | Helped |
Limitations
- Current policy explanation methods are sensitive to environmental changes and require extensive domain-specific hand engineering.
- Foundation models are prone to hallucination, which is risky in safety-critical contexts.
- Semantic explanation human-subject studies were still in progress at time of writing.
Experimental setup
Benchmarks: collaborative color-based sudoku game with robotic arm 3D AR-based human-robot Minesweeper game robotic cleaning emergency evacuation
Metrics: irreversible mistakes perceived helpfulness and usefulness trust compliance decision-making speed
Baselines: Hayes and Shah 2017 constant justifications timed-interval justifications
Builds on: RARE (Reward Augmentation and Repair through Explanation) Hayes and Shah 2017 semantic explanation method MARS
Closest papers by meaning
- Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems (NeurIPS 2024 · similarity 0.87)
- Towards Trustworthy Autonomous Systems via Conversations and Explanations (AAAI 2024 · similarity 0.86)
- Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary (NeurIPS 2024 · similarity 0.85)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.83)
- Explaining Decisions of Agents in Mixed-Motive Games (AAAI 2025 · similarity 0.83)
- Beyond Single Stationary Policies: Meta-Task Players as Naturally Superior Collaborators (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.