Research map / Multi-agent systems and games / LLM agents
LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LC-Opt, a Gymnasium/Modelica benchmark built on ORNL's Frontier liquid-cooling digital twin for RL (PPO, multi-agent, multi-head) control of cooling towers and blade groups, plus policy distillation into LLMs and decision trees with LLM-based explanations.
Problem
Liquid cooling control in HPC data centers is complex and tightly coupled, and static or rule-based controllers leave energy savings unrealized; there is no realistic open testbed for RL control of end-to-end liquid cooling.
Approach
An end-to-end, configurable Modelica digital twin of Frontier's cooling system exported as an FMU and wrapped in Gymnasium, with CT and blade-group MDPs, centralized-action batching, a multi-head policy with Dirichlet valve outputs, and distillation of RL policies into LLMs and decision trees for explanations.
Key results
- Multi-head centralized-action PPO raised blade-group temperature compliance from 76.92% (ASHRAE G36) to 95.63% while lowering cooling tower power from 237.31 kW to 206.52 kW in the medium configuration.
- Adding a heat recovery unit reduced average cooling tower power by about 21% (10.2 kW) in the N=2, m=2, C=5, B=3 setup.
- Distilled Qwen LLM controllers outperformed the PPO oracle and ASHRAE baseline on blade temperatures and cooling tower power, per the paper's ablation (exact numbers only in figure).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the centralized-action multihead RL policy instead of ASHRAE G36 | LC-Opt liquid cooling control · Unseen larger configurations | Dblade,avg (%): 71.92 -> 96.28; 68.24 -> 86.19; 75.31 -> 94.07; 83.08 -> 92.61 | Helped |
| Use the centralized-action multihead RL policy instead of ASHRAE G36 | LC-Opt liquid cooling control · Unseen larger configurations | Cooling Tower power (kW): 390.26 -> 388.84; 548.61 -> 560.72; 922.7 -> 871.26; 1381.92 -> 1109.84 | Mixed |
| Replace baseline Cooling Tower control with RL while retaining BG baseline (ablation) | LC-Opt PPO control · N=2, m=2, C=5, B=3 | Cooling Tower average power (kW): 237.31 -> 246.46 | Hurt |
| Add BG RL without valve-level control (ablation) | LC-Opt PPO control · N=2, m=2, C=5, B=3 | Dblade,avg (%): 76.92 -> 64.91 | Hurt |
| Add valve-level control to BG RL (ablation) | LC-Opt PPO control · N=2, m=2, C=5, B=3 | Dblade,avg (%): 64.91 -> 77.13 | Helped |
| Use decentralized multiagent actions (ablation) | LC-Opt PPO control · N=2, m=2, C=5, B=3 | Dblade,avg (%): 77.13 -> 78.24 | Helped |
| Centralize action execution (ablation) | LC-Opt multiagent PPO control · N=2, m=2, C=5, B=3 | Dblade,avg (%): 78.24 -> 90.46 | Helped |
| Add a multihead policy (ablation) | LC-Opt centralized-action multiagent PPO control · N=2, m=2, C=5, B=3 | Dblade,avg (%): 90.46 -> 95.63 | Helped |
| Add a multihead policy (ablation) | LC-Opt centralized-action multiagent PPO control · N=2, m=2, C=5, B=3 | Cooling Tower average power (kW): 207.37 -> 206.52 | Helped |
| Add the heat recovery unit (ablation) | LC-Opt multihead policy · Medium-sized data center (N=2, m=2, C=5, B=3) | Cooling Tower average power: approximately 21% (10.2kW average evaluated over 17 hrs) | Helped |
| Distill the trained RL policy into a fine-tuned LLM (ablation) | Cooling Tower and Blade Group controllers · LC-Opt policy evaluation | Average blade temperature and Cooling Tower power | Helped |
| Use VIPER sample weighting instead of a uniformly weighted tree (ablation) | Distilled LC-Opt decision trees · Cooling Tower policy | Average episode reward | Helped |
Surprising findings
- The decision-tree distillation of the blade-group valve policy performed poorly (R2 0.744), attributed to difficulty sampling the Dirichlet distribution.
- Under SAC, the centralized multi-head configuration did not improve over earlier cases on temperature compliance (84.33% vs 44.61% for a CT-only case, with mixed trends).
Limitations
- Benchmark lacks chip-level thermal modeling for next-generation GPUs and hybrid air-liquid cooling.
- Evaluation covers only limited weather conditions and AI workload patterns.
- Blade Group 1 sometimes runs slightly elevated temperatures due to reduced valve actuation.
Experimental setup
Datasets: Oak Ridge cold-day heat and weather dataset (input_04-07-24.csv)
Benchmarks: LC-Opt (Frontier digital twin)
Metrics: blade-group temperature compliance percentage cooling tower power consumption cabinet cooling power episode reward
Baselines: ASHRAE Guideline 36 trim-and-respond Soft Actor-Critic Decentralized multi-agent RL Few-shot prompted base LLMs
Builds on: PPO VIPER QLoRA ASHRAE Guideline 36 ExaDigiT Frontier digital twin
Closest papers by meaning
- Hierarchical Multi-Agent Framework for Carbon-Efficient Liquid-Cooled Data Center Clusters (AAAI 2025 · similarity 0.92)
- SustainDC: Benchmarking for Sustainable Data Center Control (NeurIPS 2024 · similarity 0.90)
- Data Center Cooling System Optimization Using Offline Reinforcement Learning (ICLR 2025 · similarity 0.89)
- Carbon Footprint Reduction for Sustainable Data Centers in Real-Time (AAAI 2024 · similarity 0.87)
- Sustainability of Data Center Digital Twins with Reinforcement Learning (AAAI 2024 · similarity 0.87)
- DCcluster-Opt: Benchmarking Dynamic Multi-Objective Optimization for Geo-Distributed Data Center Workloads (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.