Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Enhancing Human Experience in Human-Agent Collaboration: A Human-Centered Modeling Approach Based on Positive Human Gain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RLHG, a human-centered RL method that fine-tunes a game agent to boost human partners' goal achievement in Honor of Kings using a human-primitive-value baseline and a positive human-gain network, validated in real human-agent tests.
Problem
Game AI agents are trained in a self-centered way to maximize their own task reward (winning), which can degrade the experience of human teammates; the goal is to make collaborative agents enhance human experience while keeping their task ability.
Approach
Model human experience as human goal rewards and use a human primitive value (human return when teaming with the pre-trained agent) as a baseline; the agent is rewarded only for positive human gains above this baseline, combined with its original advantage in a policy-gradient update.
Key results
- In human-agent tests, RLHG raised high-level participants' MVP score from 8.86 to 10.28 versus Wukong, while win rate fell from 52% to 46.7%.
- General-level participants with RLHG reached MVP score 9.1 versus 7.44 with Wukong, exceeding the primitive performance of high-level participants.
- Participants rated RLHG higher on gaming experience and overall preference, with the paper concluding the enhancement gain outweighs the win-rate drop.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Wukong with RLHG | Wukong with high-level human partners · Honor of Kings human-agent test, high-level participants | MVP Score: 8.86 (0.79) -> 10.28 (0.75) | Helped |
| fine-tune Wukong with RLHG | Wukong with high-level human partners · Honor of Kings human-agent test, high-level participants | Highlight Times: 0.53 (0.21) -> 0.87 (0.29) | Helped |
| fine-tune Wukong with RLHG | Wukong with high-level human partners · Honor of Kings human-agent test, high-level participants | Participation Rate: 0.46 (0.11) -> 0.58 (0.09) | Helped |
| fine-tune Wukong with RLHG | Wukong with high-level human partners · Honor of Kings human-agent test, high-level participants | Resource Quantity: 5.3 (2.87) -> 6.28 (2.71) | Helped |
| fine-tune Wukong with RLHG | Wukong with high-level human partners · Honor of Kings human-agent test, high-level participants | Win Rate: 52% -> 46.7% | Hurt |
| fine-tune Wukong with RLHG | Wukong with general-level human partners · Honor of Kings human-agent test, general-level participants | MVP Score: 7.44 (0.71) -> 9.1 (0.61) | Helped |
| fine-tune Wukong with RLHG | Wukong with general-level human partners · Honor of Kings human-agent test, general-level participants | Highlight Times: 0.37 (0.349) -> 0.75 (0.253) | Helped |
| fine-tune Wukong with RLHG | Wukong with general-level human partners · Honor of Kings human-agent test, general-level participants | Participation Rate: 0.41 (0.11) -> 0.59 (0.05) | Helped |
| fine-tune Wukong with RLHG | Wukong with general-level human partners · Honor of Kings human-agent test, general-level participants | Resource Quantity: 4.98 (2.73) -> 5.8 (2.78) | Helped |
| fine-tune Wukong with RLHG | Wukong with general-level human partners · Honor of Kings human-agent test, general-level participants | Win Rate: 34% -> 30% | Hurt |
| use a larger human-goal weight α (ablation) | RLHG · Honor of Kings human model-agent test | human-goal performance | Helped |
| use a larger human-goal weight α (ablation) | RLHG · Honor of Kings human model-agent test | Win Rate | Hurt |
Surprising findings
- The HRE baseline, which directly adds human rewards, drops sharply on the task goal due to excessive following behavior, showing human-agent credit assignment problems.
- Participants consistently rated the high MVP score goal above the task goal of game victory in the survey.
Limitations
- Win rate of participants decreases when teaming with the RLHG agent, trading off task performance for human enhancement.
- Human goals are predefined and quantified by hand-designed rewards rather than inferred; the paper does not define or infer goals accurately.
- Evaluation uses a single game (Honor of Kings) and a fixed human model for simulated tests.
Experimental setup
Datasets: Honor of Kings game environment participant survey on in-game goals
Benchmarks: Honor of Kings 5v5 human model-agent tests Honor of Kings real-world human-agent tests
Metrics: win rate MVP score highlight times participation rate resource quantity follow rate subjective preference (behavioral rationality, enhancement d
Baselines: Wukong Human Reward Enhancement (HRE)
Builds on: PPO Wukong Dec-POMDP Generalized Advantage Estimation
Closest papers by meaning
- Towards Building Human-like Smart Agents in Modern 3D Video Games (Student Abstract) (AAAI 2025 · similarity 0.87)
- CollabBench: Benchmarking and Unleashing Collaborative Ability of LLMs with Diverse Players via Proactive Engagement (ICML 2026 · similarity 0.86)
- Learning to Cooperate with Humans using Generative Agents (NeurIPS 2024 · similarity 0.86)
- An Evolutionary Perspective on AI Alignment (Student Abstract) (AAAI 2025 · similarity 0.86)
- M³HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Quality (ICML 2025 · similarity 0.86)
- Maia-2: A Unified Model for Human-AI Alignment in Chess (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.