Research map / Multi-agent systems and games / LLM agents

OMAC: A Holistic Optimization Framework for LLM-Based Multi-Agent Collaboration

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes OMAC, a supervised contrastive-reasoning framework that optimizes five functional and structural dimensions of LLM multi-agent systems, individually and jointly, using a Semantic Initializer and a Contrastive Comparator.

Problem

LLM-based multi-agent systems are mostly handcrafted, and existing optimizers either address only agent functionality or only collaboration structure, often without supervised signals from training data.

Approach

Two LLM actors: a Semantic Initializer generates an initial pool of agents or controllers, and a Contrastive Comparator reasons over a positive-negative pair sampled by training-set scores to produce refined candidates; dimensions are optimized individually and then jointly in an iterative loop.

Key results

• Single-dimension optimization on code generation reaches up to 89.25% Pass@1 (Fun-1.4) versus 85.74% for the best baseline DyLAN.
• On MMLU general reasoning, Str-3 reaches 73.18% accuracy versus 70.06% for the best baseline AFlow.
• On MATH, joint optimization of two dimensions improves performance from 2.9% to 9.6% gain over the base MAS, per the paper's Figure.

Surprising findings

• Simultaneously optimizing multiple dimensions in one contrastive step yields smaller gains and higher variance than iterative one-dimension-at-a-time optimization.
• The Semantic Initializer alone (OMAC-C, no contrastive reasoning) already outperforms DyLAN on arithmetic reasoning across all dimensions.

Limitations

• Experiments use very small subsets (about 140-180 questions per task) and a single base LLM, gpt-3.5-turbo-1106.
• Iterative joint optimization scales poorly with the number of dimensions combined.
• Reported gains are modest in absolute terms on MATH (around 33-35% accuracy).

Experimental setup

Datasets: HumanEval MMLU MATH

Benchmarks: HumanEval MMLU MATH

Metrics: Pass@1 accuracy

Baselines: CodeT AgentVerse ADAS AFlow DyLAN Single Execution LLM Debate LLM-Blender Self-Consistency

Builds on: DyLAN ADAS AFlow

Closest papers by meaning

• AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.88)
• Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies (ICLR 2026 · similarity 0.88)
• MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems (ICML 2026 · similarity 0.88)
• Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration (ICLR 2026 · similarity 0.88)
• ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration (ICLR 2025 · similarity 0.88)
• AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.88)