Research map / Multi-agent systems and games / LLM agents

ALSO: Adversarial Online Strategy Optimization for Social Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ALSO, an online adversarial-bandit framework that selects persona-appended social strategy instructions per turn using EXP3-style exponential weights and a neural surrogate reward predictor, evaluated on Sotopia and Sotopia-Hard with a frozen LLM.

Problem

LLM social agents rely on static personas and offline or stationary optimization methods that break down in non-stationary multi-turn social simulations with co-adapting opponents.

Approach

Casts per-turn social strategy-instruction selection as an adversarial multi-armed bandit solved with exponential weights and score decay, and uses a lightweight neural surrogate over frozen embeddings to predict arm scores from dialogue context so that sparse feedback generalizes across semantically related strategies while the LLM stays frozen.

Key results

• On Sotopia-Hard, ALSO raises Overall from 3.02 (Vanilla) to 3.53 (+16.60%) and beats the best baseline INSTINCT (3.43) by +2.92%.
• Relationship on Sotopia-Hard improves from 1.32 to 2.43 (+83.79% over Vanilla).
• On Sotopia-All, ALSO reaches Overall 3.89 versus 3.85 for the best baseline, a small +0.98% margin.

Surprising findings

• On Sotopia-All, ALSO's goal score is slightly below the best baseline (8.50 vs 8.51), so gains are concentrated in relationship and overall metrics.
• The two-layer MLP surrogate used in the main results is outperformed by a 1-layer MLP in the appendix ablation, which the authors attribute to overfitting.

Limitations

• The paper explicitly does not claim a formal regret guarantee; adversarial bandit framing is used as a design rationale.
• Main results rely on a single sampled episode per scenario and small margins over baselines on Sotopia-All, with unclear statistical support.
• Strategy pool is fixed at 12 handcrafted and paraphrased strategies; larger pools degrade performance due to exploration budget.

Experimental setup

Datasets: Sotopia Sotopia-Hard

Benchmarks: Sotopia Sotopia-Hard

Metrics: Goal Relationship Knowledge Overall score

Baselines: Vanilla EvoPrompt OPRO INSTINCT Sotopia-Ω AMPO

Builds on: EXP3 Sotopia EvoPrompt OPRO INSTINCT

Closest papers by meaning

• Scaling Inference-Time Computation via Opponent Simulation: Enabling Online Strategic Adaptation in Repeated Negotiation (ICML 2026 · similarity 0.88)
• Adaptive Social Learning via Mode Policy Optimization for Language Agents (ICLR 2026 · similarity 0.88)
• Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.87)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning (ICML 2024 · similarity 0.86)
• AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making (NeurIPS 2024 · similarity 0.86)
• Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization (ICML 2026 · similarity 0.86)