Physics-guided deep reinforcement learning for optimized data center cooling and waste heat recovery utilizing aquifer thermal energy storage

Yingbo Zhang, Zixuan Wang, Konstantin Filonenko, Dominik Franjo Dominković and Shengwei Wang

Applied Energy, 2026, vol. 402, issue PB, No S0306261925017143

Abstract: A critical challenge in sustainable data center operations lies in resolving the mismatch between escalating cooling demands and waste heat utilization potential. Conventional approaches address cooling and heat recovery as separate processes, incurring systemic inefficiencies. This study develops a physics-guided deep reinforcement learning (DRL) framework that synergistically optimizes Aquifer Thermal Energy Storage (ATES) for data center cooling and waste heat recovery for nearby office buildings via heat pumps. By incorporating domain knowledge into the reward function design, the proposed approach effectively addresses delayed rewards in long-term ATES thermal balance and enables effective agent training with limited datasets. Multiple advanced DRL agents, such as DQN and D3QN, are trained to control the operation of the integrated energy systems, with dual objectives of minimizing energy consumption and maintaining annual ATES thermal balance. Results demonstrate that the both D3QN and Double DQN algorithms perform well, reducing annual energy consumption by approximately 53 % while also maintaining ATES balance within 4 %. Furthermore, the system achieves a remarkable power usage effectiveness (PUE) of 1.177, representing a 9.5 % improvement over conventional systems. Additionally, system validation via Dymola simulations demonstrates that groundwater temperatures return to initial conditions (±0.5 °C) after annual cycling. The developed framework establishes a generalizable methodology for AI-driven optimization of sustainable data center cooling systems integrated with waste heat recovery.

Keywords: Data center cooling; Waste heat recovery; Aquifer thermal energy storage; Deep reinforcement learning; Control optimization (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261925017143
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:402:y:2026:i:pb:s0306261925017143

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2025.126984

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().