Research map / Multi-agent systems and games / LLM agents

Estimating the Empowerment of Language Model Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EELMA, a contrastive (InfoNCE) estimator of effective empowerment for LM agents from text trajectories, and shows empowerment correlates with task reward across textual games, WebArena, and tau-bench.

Problem

Scalable, goal-agnostic evaluation of LM agents is needed because goal-centric benchmarks are costly and miss open-ended capabilities; classical empowerment estimators fail for high-dimensional, variable natural-language observations.

Approach

Estimate empowerment (conditional mutual information between actions and future observations) using InfoNCE over compact pretrained-text embeddings with a LoRA-adapted encoder, so that linguistic variability is absorbed by the embedding space and no goals or rewards are needed by the metric.

Key results

• On WebArena, empowerment correlates with discounted reward (Spearman R_s=0.83-0.94 in GitLab, Reddit, Shopping Admin domains); pooled R_s=0.39 across 16 model-domain pairs vs. -0.04 to 0.16 for diversity baselines.
• On tau-bench airline, empowerment scales monotonically with Qwen2.5 size (1.5B 0.59, 7B 0.71, 32B 0.74 bits) matching reward ranking; the retail domain shows an inversion at floor performance.
• Disabling chain-of-thought reduces Gridworld empowerment by 99% (0.19 to 0.01 bits) and ToH empowerment by 65% (0.29 to 0.09 bits).

Surprising findings

• In tau-bench retail, where all models perform near floor, smaller models get higher action-state MI by hallucinating diverse invalid tool calls, so empowerment inverts.
• Qwen2.5 navigates to external real websites in Reddit tasks, inflating empowerment despite task failure (jailbreaking behavior).

Limitations

• Empowerment measures option count and control, not whether actions advance the task; it fails when reasoning is the bottleneck (WebArena Shopping).
• Finite-K InfoNCE leaves residual bias, so negative empowerment values can arise and the estimator is an approximation.
• Uninformative when all models operate near floor performance (tau-bench retail).

Experimental setup

Datasets: WebArena tau-bench Gridworld (text-based, self-constructed) Tower of Hanoi (text-based, self-constructed)

Benchmarks: Gridworld Tower of Hanoi WebArena tau-bench

Metrics: Spearman correlation RMSE (bits) mean discounted reward success rate t-test p-value

Baselines: direct (counting-based) estimation visitation entropy action entropy state coverage trajectory length unique-state ratio

Builds on: InfoNCE empowerment (Klyubin et al.) Myers et al. empowerment-reward connection LoRA Blahut-Arimoto algorithm

Closest papers by meaning

• Evaluating Language Model Agency Through Negotiations (ICLR 2024 · similarity 0.87)
• From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills (NeurIPS 2026 · similarity 0.87)
• Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (ICLR 2025 · similarity 0.86)
• The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement (NeurIPS 2025 · similarity 0.86)
• TALES: Text-Adventure Learning Environment Suite (NeurIPS 2026 · similarity 0.86)
• Abstract Counterfactuals for Language Model Agents (NeurIPS 2025 · similarity 0.86)