Research map / Reinforcement learning / Visual and representation RL
Sustainability of Data Center Digital Twins with Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DCRL-Green, an open-source OpenAI Gym-based multi-agent RL environment and digital twin for co-designing and controlling data center components (HVAC, load shifting, battery storage) to reduce carbon footprint.
Problem
Data center energy use and carbon emissions are growing with ML demand, but existing RL environments optimize individual DC components (HVAC setpoints, load shifting, battery scheduling) in isolation rather than jointly.
Approach
A modular, configurable data center digital twin exposed as a multi-agent Gym environment (built on RLlib) that jointly controls flexible load shifting, HVAC setpoints, and battery storage, with ASHRAE G36 as the rule-based reference and single- and multi-agent RL baselines.
Key results
- Single-agent RL HVAC setpoint control (PPO) reported about 7% carbon emission reduction; PPO carbon footprint 13726.56 tonnes CO2 vs ASHRAE 14839.00.
- Multi-agent MADDPG reported about 13% carbon emission reduction relative to ASHRAE G36 control.
- A2C reached 13878.30 tonnes CO2 carbon footprint, compared with 14839.00 for ASHRAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPO instead of ASHRAE Guideline36 for HVAC setpoint control | Data center HVAC control · Single-agent HVAC control | Carbon footprint: 14839.00 -> 13726.56 | Helped |
| Use A2C instead of ASHRAE Guideline36 for HVAC setpoint control | Data center HVAC control · Single-agent HVAC control | Carbon footprint: 14839.00 -> 13878.30 | Helped |
| Use multiagent MADDPG instead of ASHRAE G36 control | Data center control · Arizona | Energy reduction (%): 13.4 ± 0.48 | Helped |
| Use multiagent MADDPG instead of ASHRAE G36 control | Data center control · Arizona | Carbon footprint reduction (%): 13.38 ± 0.62 | Helped |
| Use multiagent MADDPG instead of ASHRAE G36 control | Data center control · New York | Energy reduction (%): 13.01 ± 0.12 | Helped |
| Use multiagent MADDPG instead of ASHRAE G36 control | Data center control · New York | Carbon footprint reduction (%): 12.77 ± 0.11 | Helped |
| Use multiagent MADDPG instead of ASHRAE G36 control | Data center control · Washington | Energy reduction (%): 13.27 ± 0.06 | Helped |
| Use multiagent MADDPG instead of ASHRAE G36 control | Data center control · Washington | Carbon footprint reduction (%): 13.51 ± 0.05 | Helped |
Limitations
- Evaluation limited to a default setup with HVAC setpoint control results reported in a short paper.
- Holistic multi-component optimization is described as a goal; results shown mainly for HVAC control.
Experimental setup
Datasets: data center workload profiles weather data grid carbon intensity data
Benchmarks: DCRL-Green default setup
Metrics: energy footprint carbon footprint (tonnes CO2) percentage reduction vs ASHRAE G36
Baselines: PPO A2C ASHRAE G36 MADDPG
Builds on: OpenAI Gym RLlib MADDPG EnergyPlus
Closest papers by meaning
- Hierarchical Multi-Agent Framework for Carbon-Efficient Liquid-Cooled Data Center Clusters (AAAI 2025 · similarity 0.92)
- Carbon Footprint Reduction for Sustainable Data Centers in Real-Time (AAAI 2024 · similarity 0.91)
- SustainDC: Benchmarking for Sustainable Data Center Control (NeurIPS 2024 · similarity 0.91)
- Data Center Cooling System Optimization Using Offline Reinforcement Learning (ICLR 2025 · similarity 0.87)
- LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers (NeurIPS 2025 · similarity 0.87)
- DCcluster-Opt: Benchmarking Dynamic Multi-Objective Optimization for Geo-Distributed Data Center Workloads (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.