Research map / Multi-agent systems and games / LLM agents

HACO: Hedged Agent Computing for Reliable LLM Systems

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HACO, a runtime control scheme that adaptively selects a reliability-constrained hedge set of role-specific LLM agent instances (across execution zones) per role invocation, and refines candidate and link profiles from executed traces.

Problem

LLM agent systems assume a stable execution environment, but the same role request yields different latency, failure probability, and quality depending on which agent instance (LLM plus region/network) executes it, causing reliability failures in deployment.

Approach

Treat each role invocation as a three-axis (role type, LLM, execution environment) decision: rank candidates by optimistic utility (quality upper bound times execution success over latency discount), then add candidates with conservative reliability estimates until the cumulative success probability reaches a target tau; harvest traces from all hedge-set executions to update Beta quality posterior

Key results

• Under global-link and candidate degradation, BestOne's all-failed rate rises by 72.8 and 67.4 percentage points while HACO stays stable and reduces Global selections by 80 and 40 points.
• Raising reliability target tau from 0.70 to 0.95 lowers step failure rate from 1.87% to 0.17% and raises average score from 53.77 to 58.86.
• Ablation on MatplotBench: removing the uncertainty bonus drops score to 51.61 from 58.86, removing environment-aware modeling to 53.00, and replacing conservative stopping to 53.90.

Surprising findings

• Evaluator degradation (weak Llama-3.1-8B judge) has only a minor effect on HACO, suggesting it works with a weak LLM-as-judge.

Limitations

• Experiments use a simulated network and simulated failure injection rather than primarily real deployments; real Azure deployment is only a validation.
• Assumes conditional independence across candidates when computing hedge-set failure probability.
• Latency is evaluated with an LLM-judge quality signal and a lightweight simulator that abstracts transport-level behavior.

Experimental setup

Datasets: DSBench InfiAgent-Bench MatplotBench

Benchmarks: DSBench InfiAgent-Bench MatplotBench

Metrics: effectiveness score strict success all-failed rate token usage latency

Baselines: Random BestOne MoA-style Best-of-N Fixed-zone Local Fixed-zone Regional Fixed-zone Global

Builds on: DATAWISE MetaGPT Mixture-of-Agents NIST Net / Linux NetEm

Closest papers by meaning

• Rethinking the Reliability of Multi-agent System: A Perspective from Byzantine Fault Tolerance (AAAI 2026 · similarity 0.88)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs (NeurIPS 2026 · similarity 0.87)
• AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.87)
• AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.87)
• PACE: Two-Timescale Self-Evolution for Small Language Model Agents (NeurIPS 2026 · similarity 0.87)
• How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.86)