Research map / Multi-agent systems and games / Human-AI and goal-conditioned

Towards Building Human-like Smart Agents in Modern 3D Video Games (Student Abstract)

AAAI 2025 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Human-AI coordination agents

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a hierarchical RL framework that learns a VAE-style behavior model and expert prior via imitation learning at the low level and a PPO high-level policy at the top, to build human-like agents in PUBG Mobile.

Problem

RL agents in complex 3D games achieve victory but often behave unlike humans (e.g., collisions, unnecessary reversing), and reward shaping or imitation learning alone each have limitations.

Approach

A hierarchical framework where a low-level latent-intention behavior model (ELBO-trained encoder/decoder over fixed-length action sequences) and a state-conditioned expert prior are learned by imitation, and a high-level PPO policy outputs intention vectors, regularized by KL to the expert prior.

Key results

• Hier-RL achieves 100% win rate vs built-in agents, compared with 95% for PPO, 87.9% for IBC and 82.4% for BC.
• Hier-RL reduces reversing counts to 23.6 versus 353.3 for PPO and 1210.0 for BC.
• Hier-RL increases valid sliding to 48.9 versus 31.3 for PPO.

Measured effects reported in the paper

Limitations

• Relies on offline demonstration data for imitation learning
• Early-stage work tested in a single game environment
• Human-likeness assessed via subjective categorization by 30 players/developers

Experimental setup

Datasets: PUBG Mobile human player demonstrations

Benchmarks: PUBG Mobile adversarial combat tests vs built-in agents (1,000 rounds)

Metrics: win rate kill count hit count counts of undesired behaviors (reversing, against shelter, s fighting move/prone counts valid sliding

Baselines: Behavior Cloning Implicit Behavior Cloning PPO with human-like reward shaping

Builds on: PPO ELBO variational autoencoder Implicit Behavior Cloning Pertsch et al. skill-prior hierarchical RL

Closest papers by meaning

• Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Games (NeurIPS 2024 · similarity 0.87)
• Enhancing Human Experience in Human-Agent Collaboration: A Human-Centered Modeling Approach Based on Positive Human Gain (ICLR 2024 · similarity 0.87)
• Learning Strategy Representation for Imitation Learning in Multi-Agent Games (AAAI 2025 · similarity 0.86)
• Learning Human-Like RL Agents Through Trajectory Optimization With Action Quantization (NeurIPS 2025 · similarity 0.85)
• Integrating Suboptimal Human Knowledge with Hierarchical Reinforcement Learning for Large-Scale Multiagent Systems (NeurIPS 2024 · similarity 0.85)
• GUIDE: Real-Time Human-Shaped Agents (NeurIPS 2024 · similarity 0.84)