Research map / Multi-agent systems and games / LLM agents

Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning

CVPR 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CrossHA, a Minecraft VLM agent trained with SFT, single-turn GRPO, and multi-turn GRPO to learn step-level selection among heterogeneous action spaces (raw, language, motion, grounding, latent).

Problem

Existing native agents are confined to a fixed action space (API, GUI, robotic commands, or Minecraft raw inputs), while the optimal action granularity varies across tasks and steps within a trajectory.

Approach

Treat action-space selection as a learnable policy decision: a three-stage pipeline (mixed-space SFT, action-space-agnostic single-turn GRPO with a parser-based reward, then multi-turn GRPO with episodic success and a token-length penalty) lets one model switch among heterogeneous action spaces within a trajectory.

Key results

• CrossHA reaches 54.6% ASR across all 800+ tasks, versus 42.2% for Game-TARS and 33.8% for UI-TARS-1.5 as reported in the main table.
• CrossHA achieves 78.8% ASR on Craft Items, the state of the art in that category, and 94.7% FT on Mine Blocks.
• On OOD tasks, CrossHA reaches 49.1% overall success versus 42.4% for RawHA-RL, and removing STRL drops it to 39.7%.

Surprising findings

• Removing the single-turn RL stage lowers Craft Items OOD success from 78.8% to 58.0%, showing the warm start matters for generalization.
• Single-space RL agents achieve very high in-distribution Craft Items success (RawHA-RL 96.2%) but drop sharply on OOD tasks.

Limitations

• Evaluated only in Minecraft; real-world robotics and safety/latency not addressed.
• Multi-turn RL is computationally costly (1.3M images, 80+ iterations with 6,400 interactions each).

Experimental setup

Datasets: VPT data contractor-collected Minecraft trajectories Minecraft VQA and captioning datasets OpenHA benchmark tasks

Benchmarks: OpenHA benchmark suite (800+ Minecraft tasks)

Metrics: Finished Tasks (FT) Average Success Rate (ASR) success rate

Baselines: VPT STEVE-1 ROCKET-1 JARVIS-VLA OpenHA Game-TARS UI-TARS-1.5 GroundingHA MotionHA LatentHA

Builds on: GRPO OpenHA Qwen2-VL VPT MineCLIP

Closest papers by meaning

• WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents (NeurIPS 2025 · similarity 0.86)
• APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay (NeurIPS 2025 · similarity 0.86)
• OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents (NeurIPS 2024 · similarity 0.86)
• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning (ICML 2026 · similarity 0.86)
• Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization (ICML 2026 · similarity 0.86)
• A Domain-Independent Agent Architecture for Adaptive Operation in Evolving Open Worlds (AAAI 2025 · similarity 0.86)