A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems

Lissy Langer and Thomas Volling

Applied Energy, 2022, vol. 327, issue C, No S0306261922012776

Abstract: Buildings are one of the main drivers of global energy consumption and CO2 emissions. Efficient energy management systems will have to integrate renewable energy sources with heating and/or cooling to mitigate climate change. In this study, we analyze the potential of deep reinforcement learning (DRL) to control a smart home with a modulating air-to-water heat pump, a photovoltaic system, a battery energy, and a thermal storage system for floor heating and hot water supply. We transform a mixed-integer linear program (MILP) into a DRL implementation. In our numerical analysis, we compare our results based on the deep deterministic policy gradient (DDPG) algorithm to the theoretical upper bound of the model predictive control (MPC) result under full information, as well as a practice-oriented rule-based benchmark. We show that our proposed DRL implementation outperforms the rule-based approach and achieves a self-sufficiency of 75% with only limited comfort violations. Analyzing different DRL formulations, we conclude that domain knowledge is key to formalizing an efficient decision problem with stable results. Our input data and models, developed using the Julia programming language, are available open source.

Keywords: Home energy management; Building energy management; Heat pump; Photovoltaics (PV); Reinforcement learning; Deep deterministic policy gradient (DDPG) (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (9)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261922012776
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:327:y:2022:i:c:s0306261922012776

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2022.120020

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().