Research map / Reinforcement learning / Visual and representation RL
Hierarchical Multi-Agent Framework for Carbon-Efficient Liquid-Cooled Data Center Clusters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Green-DCC, an RL-based hierarchical multi-agent simulator that jointly shifts workloads geographically across data center clusters and temporally within data centers while controlling liquid cooling, reporting CO2 reductions with PPO.
Problem
Reducing the carbon footprint of cloud computing by coordinating workload distribution across geographically dispersed data center clusters together with temporal workload shifting and liquid/HVAC cooling control within each data center.
Approach
A two-level hierarchy in which a top-level RL agent performs geographic load shifting across data centers based on carbon intensity and weather, and low-level agents at each data center perform temporal load shifting via a deferred task queue and control liquid cooling flow and setpoints, trained jointly as hierarchical RL.
Key results
- Hierarchical RL with PPO achieved the lowest CO2 at 3435 ± 5 versus 3845 ± 3 for the baseline in the load-shift-only setting.
- Top-level RL alone reached 3515 ± 3 and Top+Low-level RL reached 3512 ± 3.
- PPO outperformed A2C (3601 ± 3) and APPO (3608 ± 3) under hierarchical RL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hierarchical RL with all agents trained simultaneously instead of the baseline | Green-DCC · Data center cluster, load shifting | CO2 emissions (tons): 3845 ± 3 -> 3435 ± 5 | Helped |
| Add top-level geographic workload distribution (ablation) | Green-DCC · Data center cluster, load shifting | CO2 emissions (tons): 3845 ± 3 -> 3515 ± 3 | Helped |
| Add temporal load shifting with a pre-trained low-level agent to top-level geographic distribution (ablation) | Green-DCC · Data center cluster, load shifting | CO2 emissions (tons): 3515 ± 3 -> 3512 ± 3 | Helped |
| Train all agents simultaneously in a hierarchical RL setup instead of using a pre-trained low-level agent (ablation) | Green-DCC · Data center cluster, load shifting | CO2 emissions (tons): 3512 ± 3 -> 3435 ± 5 | Helped |
| Use PPO instead of A2C (ablation) | Hierarchical RL · Data center cluster, load shifting | CO2 emissions (tons): 3601 ± 3 -> 3435 ± 5 | Helped |
| Use PPO instead of APPO (ablation) | Hierarchical RL · Data center cluster, load shifting | CO2 emissions (tons): 3608 ± 3 -> 3435 ± 5 | Helped |
Limitations
- Hierarchical RL is the most complex to train.
- Results reported only for load-shift configurations in the presented tables.
- Evaluated on a simulated three-site cluster.
Experimental setup
Datasets: simulated data center cluster with New York, California, and Georgia locations (weather, grid carbon intensity, workload
Benchmarks: Green-DCC simulation environment
Metrics: CO2 emissions (tons)
Baselines: Baseline (no RL) Top-Level RL Top+Low-Level RL A2C APPO
Builds on: PPO A2C APPO
Closest papers by meaning
- Carbon Footprint Reduction for Sustainable Data Centers in Real-Time (AAAI 2024 · similarity 0.92)
- LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers (NeurIPS 2025 · similarity 0.92)
- Sustainability of Data Center Digital Twins with Reinforcement Learning (AAAI 2024 · similarity 0.92)
- SustainDC: Benchmarking for Sustainable Data Center Control (NeurIPS 2024 · similarity 0.91)
- DCcluster-Opt: Benchmarking Dynamic Multi-Objective Optimization for Geo-Distributed Data Center Workloads (NeurIPS 2025 · similarity 0.90)
- Data Center Cooling System Optimization Using Offline Reinforcement Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.