Research map / Multi-agent systems and games / Cooperative multi-agent RL
SustainDC: Benchmarking for Sustainable Data Center Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SustainDC, a Python benchmark suite of customizable data center environments (workload scheduling, cooling setpoint, battery control) for evaluating single- and multi-agent RL algorithms on carbon footprint reduction.
Problem
Data center operations consume large amounts of energy and carbon; existing RL environments for building/DC control rely on precompiled EnergyPlus/Modelica binaries and do not jointly model workload scheduling, cooling, and battery control for multi-agent RL.
Approach
A pure-Python, end-user customizable simulator that couples three cascading control problems (load shifting, CRAH cooling setpoint, battery charge/discharge) as heterogeneous multi-agent RL tasks with configurable rewards, including a collaborative reward-sharing mechanism controlled by a weighting parameter alpha.
Key results
- Multi-agent IPPO with all three agents achieves lower energy and water use than single-agent baselines in the New York setting, with CFP comparable across experiments (about 176-182 kgCO2 across MARL variants in New York).
- In Arizona, MAPPO/HAPPO variants and HASAC show significant variance across regions; HASAC outperforms others in Arizona while HAPPO performs better in most other locations.
- Setting the collaborative reward weight alpha=0.8 gave slightly lower CFP and task queue than alpha=1.0 (no collaboration) in the IPPO ablation (176.2 vs 176.3 kgCO2, 9.5 vs 12.5 queue).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a cooling RL controller instead of ASHRAE Guideline 36 | SustainDC data center control · SustainDC | CFP | Helped |
| Operate workload, cooling, and battery RL agents simultaneously | SustainDC · SustainDC | IT energy | Helped |
| Use collaborative reward sharing with α = 0.8 rather than no collaboration (α = 1.0) (ablation) | IPPO · SustainDC | CFP (kgCO2): 176.3 -> 176.2 | Helped |
| Use collaborative reward sharing with α = 0.8 rather than α = 0.1 (ablation) | IPPO · SustainDC | CFP (kgCO2): 176.4 -> 176.2 | Helped |
| Replace independent-critic IPPO with shared-critic MAPPO or HAPPO | Multi-agent SustainDC control · New York, Georgia, and California | CFP: New York: 179.6 -> 176.4; Georgia: 265.4 -> 263.4; California: 170.0 -> 159.3 | Mixed |
| Use HAPPO rather than MAPPO or HAA2C | Heterogeneous multi-agent control · SustainDC locations | CFP | Mixed |
| Use HASAC rather than the other tested algorithms | Heterogeneous multi-agent control · Arizona | CFP: 404.6 | Helped |
| Add reward sharing from other agents to the IPPO reward (ablation) | IPPO · SustainDC | CFP: 176.3 -> 176.2 | No clear effect |
Surprising findings
- The battery agent failed to learn when to charge from the grid and discharge optimally, so IPPO's battery control did not exploit carbon-intensity variation.
- No single MARL algorithm performed best across all metrics and locations, and results varied strongly across regions.
Limitations
- No oracle optimum exists, making it hard to quantify the performance gap to best achievable results.
- Hyperparameters could not be exhaustively tuned for all networks.
- Reported results are from a single server with 4 threads per training agent; sim-to-real deployment is not yet demonstrated.
Experimental setup
Datasets: Alibaba cluster workload traces Google cluster workload traces EIA grid carbon intensity data EnergyPlus weather files (.epw)
Benchmarks: SustainDC
Metrics: CFP (kgCO2) HVAC energy (kWh) IT energy (kWh) task queue water usage (litre)
Baselines: IPPO MAPPO HAPPO HADQN HASAC ASHRAE Guideline 36 controller rule-based workload baseline rule-based battery baseline
Builds on: PPO IPPO MAPPO HAPPO CityLearn
Closest papers by meaning
- Carbon Footprint Reduction for Sustainable Data Centers in Real-Time (AAAI 2024 · similarity 0.92)
- Sustainability of Data Center Digital Twins with Reinforcement Learning (AAAI 2024 · similarity 0.91)
- Hierarchical Multi-Agent Framework for Carbon-Efficient Liquid-Cooled Data Center Clusters (AAAI 2025 · similarity 0.91)
- LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers (NeurIPS 2025 · similarity 0.90)
- DCcluster-Opt: Benchmarking Dynamic Multi-Objective Optimization for Geo-Distributed Data Center Workloads (NeurIPS 2025 · similarity 0.89)
- Data Center Cooling System Optimization Using Offline Reinforcement Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.