Research map / Multi-agent systems and games / LLM agents

AutoRAS: Learning Robust Agentic Systems with Primitive Representations

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoRAS, which automatically designs robust multi-agent systems by generating sequences of symbolic primitives with a GFlowNet (trajectory balance) policy driven by execution-derived safety signals and textual gradients.

Problem

Automated design of LLM agentic systems usually optimizes task accuracy and treats robustness to external adversarial attacks and internal failures as an afterthought.

Approach

Represents an agentic system as a legal sequence of structural and behavioral primitives compiled by a stack machine, and learns a sequence-generation policy with trajectory balance over a reward that combines accuracy, external/internal robustness, and cost, plus textual gradients that refine primitive prompts from execution traces.

Key results

• AutoRAS reaches 74.27% average vanilla accuracy across four datasets, the best among 11 baselines, with 72.14% under attack.
• Under attack, AutoRAS has the smallest average drop (2.13 points), versus 14.26 for AFlow and 18.43 for MaAS.
• On MMLU with GPT-4o-mini, AutoRAS scores 83.01 vanilla and 82.35 under attack, at total cost of $0.6655 versus $0.33 for MaAS.

Surprising findings

• Removing the safety signal causes the largest drop under attack on MMLU (82.35 to 71.90), while removing textual gradients has the smallest impact.
• Stronger analyzer or monitor backbones give only marginal gains, and a random-prediction monitor was tested as a stress test.

Limitations

• Robustness evaluated only on four attack types within a stated threat model, not immunity to all attacks.
• Very small training sets (6 to 119 queries per dataset) and small test sets (24 to 486) raise variance concerns.
• Cost is slightly higher than some baselines due to analyzer and monitor overhead.

Experimental setup

Datasets: MMLU MSMARCO MATH ProgramDev

Benchmarks: MMLU MSMARCO MATH ProgramDev

Metrics: accuracy executability performance drop under attack cost

Baselines: CoT Self-Consistency LLM-Debate DyLAN G-Safeguard GPTSwarm AgentPrune AFlow G-Designer MaAS

Builds on: GFlowNet trajectory balance AFlow MaAS G-Designer

Closest papers by meaning

• Automated Design of Agentic Systems (ICLR 2025 · similarity 0.87)
• AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.86)
• AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvement (NeurIPS 2025 · similarity 0.86)
• Agent Primitives: Reuseable Latent Building Blocks for Multi-Agent Systems (ICML 2026 · similarity 0.86)
• ResMAS: Resilience Optimization in LLM-based Multi-agent Systems (AAAI 2026 · similarity 0.86)
• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning (ICML 2026 · similarity 0.86)