Research map / Multi-agent systems and games / LLM agents

PerfGuard: A Performance-Aware Agent for Visual Content Generation

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: GUI agent memory and trajectories · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PerfGuard, a performance-aware LLM agent for visual content generation that scores tools on fine-grained benchmark dimensions, updates those scores from actual execution rankings, and trains the planner with stepwise preference optimization.

Problem

LLM agents assume tool calls always succeed and describe tools with coarse textual descriptions, so tool selection and planning are unreliable in visual content generation (AIGC) where tool performance varies by dimension.

Approach

Replace textual tool descriptions with a multi-dimensional performance-boundary matrix (initialized from T2I-CompBench and ImgEdit-Bench scores), select tools by task-specific dimension weights, update the matrix by comparing theoretical and observed rankings, and align the planner with SPO-style stepwise preference optimization using a self-evaluator.

Key results

• On T2I-CompBench, PerfGuard reports Color 0.8753, Spatial 0.6120 and Complex 0.5007, above the best listed baseline GenArtist (0.8482, 0.5437, 0.4499).
• On the tool-selection simulation with the complex_vel subset, the error rate falls from 77.8% (text descriptions with Qwen3-14B) to 14.2% with performance matrix plus preference updating.
• In a user study with 15 participants, 73.3% of users preferred PerfGuard's outputs as the best tool.

Surprising findings

• Text-description-based tool selection with Qwen3-14B has a 77.8% error rate, and GPT-4o only reduces it to 72.2%, suggesting LLMs struggle to interpret capability descriptions alone.
• PerfGuard does not show a large margin over baselines on OneIG alignment and text metrics, attributed to toolset limitations.

Limitations

• Relies on existing benchmark scores to initialize the performance matrix, which may not exist outside visual content generation.
• Toolset limits generation capability, so alignment and text gains are modest.
• Evaluation uses GPT-4o as judge for several metrics, and many baselines' open-weight status and sizes are unclear.

Experimental setup

Datasets: T2I-CompBench ImgEdit-Bench MS-COCO

Benchmarks: T2I-CompBench OneIG-Bench Complex-Edit

Metrics: color accuracy shape accuracy texture accuracy spatial accuracy non-spatial accuracy complex score OneIG alignment, text, reasoning, style Complex-Edit IF, PQ, IP, O DINO score CLIP score

Baselines: FLUX SD3 GoT T2I-R1 GenArtist T2I-Copilot Step1X_Edit AnySD OmniGen ICEdit

Builds on: MetaGPT Step-aware Preference Optimization (SPO) Direct Preference Optimization (DPO) CompAgent GenArtist

Closest papers by meaning

• A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks (NeurIPS 2026 · similarity 0.88)
• MMBench-GUI: A Unified Hierarchical Evaluation Framework for Multi-Platform GUI Agents (CVPR 2026 · similarity 0.86)
• Learning to Select Visual Tools from Experience (CVPR 2026 · similarity 0.86)
• Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale (ICML 2025 · similarity 0.86)
• CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization (CVPR 2026 · similarity 0.86)
• ShowUI: One Vision-Language-Action Model for GUI Visual Agent (CVPR 2025 · similarity 0.86)