An occupant-centric control framework for balancing comfort, energy use and hygiene in hot water systems: A model-free reinforcement learning approach

Amirreza Heidari, François Maréchal and Dolaana Khovalyg

Applied Energy, 2022, vol. 312, issue C, No S0306261922002744

Abstract: Occupants’ behavior is a major source of uncertainty for the optimal operation of building energy systems. The highly stochastic hot water use behavior of occupants has led to conservative operational strategies for hot water systems, that try to ensure occupants’ comfort by following energy-intensive operational approaches. Intending to integrate the occupants’ behavior into hot water systems control, this study proposes a control framework based on Reinforcement Learning, which can learn the stochastic occupants' behavior and make a balance between opposing objectives of water hygiene, comfort, and energy use. A model-free approach is implemented to ensure transferability. To achieve fast convergence on the target house while being model-free, this study proposes an offline training procedure integrating a stochastic hot water use model to mimic the hot water use behavior of occupants. The proposed framework is then evaluated using an actual hot water use and weather dataset collected over 29 weeks from a residential house in Switzerland. The performance of the proposed control framework is compared to the conventional rule-based controller as the common practice in hot water systems. While the hot water use dataset has been collected during the COVID-19 pandemic, with an abnormal schedule of occupants, results indicate that the proposed control framework could successfully learn and adapt to the occupants' behavior and achieve 23.8% of energy saving, while maintaining the occupants' comfort and water hygiene. The adaptive nature of the proposed control framework shows significant potential in reducing the discrepancy between supply and demand in hot water systems.

Keywords: Energy; Building; Reinforcement learning; Machine learning; Occupant behavior; Deep Q-learning (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (11)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261922002744
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:312:y:2022:i:c:s0306261922002744

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2022.118833

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().