Research map / Multi-agent systems and games / LLM agents

TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TowerMind, a lightweight multimodal tower-defense RTS environment with five difficulty levels, used to benchmark LLM agents (pixel, text, and structured observations) against human experts and to run Ape-X DQN and PPO baselines.

Problem

Existing RTS-game environments for LLM evaluation are computationally heavy (SC2LE) or lightweight ones lack textual observation and action interfaces, so there is no low-cost RTS-style benchmark for LLM long-term planning and decision-making.

Approach

A Unity-based tower defense game exposed via a Gym-style interface with pixel, JSON-textual and flattened structured observations, a hybrid (x, y, action type) action space, validity/error-code tracking for hallucination measurement, a level editor, and a quantitative level difficulty metric.

Key results

• Claude 3.7 Sonnet achieved the best language-only average score (0.38 normalized to human experts), and GPT-4.1 the best vision-language average (0.42), trailing humans by 62% and 58%.
• Claude 3.7 Sonnet reached 0.82 average valid action rate in vision-language mode, while Qwen2.5-VL 7B and Llama 3.2 11B fell below the random baseline on several levels.
• Both Ape-X DQN and PPO solved only simpler levels after 100M steps and remained substantially below human experts.

Surprising findings

• Llama 3.2 90B performed worse with vision input than language-only in score, while most other models improved with vision.
• LLMs' valid action rates were much closer to human experts than their scores, suggesting correctness is achieved but effectiveness lags.

Limitations

• Only zero-shot prompting evaluated; no prompt or fine-tuning strategies explored for LLMs.
• Small number of runs and levels (five levels, five seeds) with human baseline from five experts.

Experimental setup

Benchmarks: TowerMind Levels 1-5

Metrics: score valid action rate

Baselines: GPT-4.1 Gemini-2.5-Pro Claude 3.7 Sonnet Llama 3.2 90B Llama 3.2 11B Qwen2.5-VL 72B Qwen2.5-VL 7B Ape-X DQN PPO human experts random policy

Builds on: Unity ML-Agents OpenAI Gym ELF Ape-X DQN PPO

Closest papers by meaning

• BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (ICLR 2025 · similarity 0.89)
• LLM-PySC2: Starcraft II learning environment for Large Language Models (NeurIPS 2025 · similarity 0.89)
• Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search (ICLR 2025 · similarity 0.88)
• Expanding LLM Agent Boundaries with Strategy-Guided Exploration (NeurIPS 2026 · similarity 0.88)
• SmartPlay : A Benchmark for LLMs as Intelligent Agents (ICLR 2024 · similarity 0.88)
• MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs (ICLR 2026 · similarity 0.88)