Data-driven heat pump operation strategy using rainbow deep reinforcement learning for significant reduction of electricity cost

Gwangwoo Han, Hong-Jin Joo, Hee-Won Lim, Young-Sub An, Wang-Je Lee and Kyoung-Ho Lee

Energy, 2023, vol. 270, issue C

Abstract: The need for reducing carbon emissions and achieving “Net Zero” energy has made improving heat pumps’ (HPs) operational efficiency a crucial goal. However, current rule or model-based control strategies have limitations of inability to consider the entire heat production-storage-utilization cycle and inherent difficulties in achieving both high performance and generality. Here, we propose a model-free deep reinforcement learning (DRL)-based HP operation strategy that utilizes the Rainbow deep Q network algorithm to minimize electricity costs by considering thermal load demand, renewable generation, coefficient of performance (COP) of HPs, and state of charge (SOC) of thermal storage. We employ artificial neural networks to train for the regression of future load demands and COP, creating a data-driven and connectable environment with DRL. The Rainbow agent learns a creative strategy of limiting the maximum number of HP operations by increasing the SOC in advance to match future load demands. The performance of the Rainbow agent is evaluated against rule-based control in cases of future states, future uncertainty, and five-year long-term deployment. The proposed method reduces the year-round demand charge by 23.1% and the energy charge by 21.7%, resulting in a 22.2% reduction in the electricity cost.

Keywords: Deep reinforcement learning; Heat pump; Electricity cost; Rainbow deep Q network; Load demand; Renewable energy (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544223003079
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:270:y:2023:i:c:s0360544223003079

DOI: 10.1016/j.energy.2023.126913

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().