Research map / Multi-agent systems and games / LLM agents

Promoting Sustainable Web Agents: Benchmarking and Estimating Energy Consumption Through Empirical and Theoretical Analysis

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measures energy and CO2 of five open-source web agents on Mind2Web across eight GPUs and estimates energy for MindAct and the GPT-4-based LASER, arguing energy should be a benchmark metric.

Problem

Web agents built on LLMs have largely unexplored energy and CO2 costs, and energy is invisible to users and absent from benchmarks; proprietary-LLM agents cannot be benchmarked directly.

Approach

Combines empirical energy benchmarking of open-source web agents on Mind2Web (via carbontracker across eight GPUs) with a theoretical estimation method (token counts times per-token energy, derived from FLOPs and power assumptions) for agents using proprietary LLMs, validated by comparing estimate vs. measurement for MindAct.

Key results

• AutoWebGLM was the most energy-efficient agent (0.33 kWh on H100-NVL) and also had the highest average SSR (53.53), while Synatra consumed about 10x more energy (3.31 kWh) with SSR 15.85.
• Estimated MindAct energy (8.5 kWh) overestimates the benchmarked 1.22 kWh by roughly a factor of 7.
• Estimated LASER (GPT-4) energy on Mind2Web is 99.21 kWh, roughly 10 times MindAct's estimate, under conservative assumptions.

Surprising findings

• Higher energy consumption did not correspond to better results; the most energy-hungry open-source agents had the lowest SSR.
• Theoretical estimation overestimated MindAct energy by about a factor of 7, showing estimates are unreliable even for open agents.

Limitations

• GPT-4 energy estimate relies on leaked, unverified parameter counts and assumed MoE architecture and hardware utilization.
• Estimation assumes upper-bound token counts and a best-case GPU performance, making it coarse and likely an overestimate.
• Only five open-source agents and Mind2Web benchmark were benchmarked, limiting generality.

Experimental setup

Datasets: Mind2Web

Benchmarks: Mind2Web

Metrics: energy (kWh) energy per token (kWh) computation time (min) CO2 emissions (g) average step success rate

Baselines: AutoWebGLM MindAct MultiUI Synapse Synatra LASER

Builds on: Mind2Web MindAct LASER carbontracker

Closest papers by meaning

• Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (ICLR 2025 · similarity 0.86)
• REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites (NeurIPS 2025 · similarity 0.86)
• The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization (NeurIPS 2025 · similarity 0.85)
• AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Web (ICML 2026 · similarity 0.85)
• TimeWarp: Evaluating Web Agents by Revisiting the Past (NeurIPS 2026 · similarity 0.85)
• EcoGEO: Trajectory-Aware Evidence Ecosystems for Web-Enabled LLM Search Agents (NeurIPS 2026 · similarity 0.84)