Scalable energy management approach of residential hybrid energy system using multi-agent deep reinforcement learning

Zixuan Wang, Fu Xiao, Yi Ran, Yanxue Li and Yang Xu

Applied Energy, 2024, vol. 367, issue C, No S0306261924007979

Abstract: Deploying renewable energy and implementing smart energy management strategies are crucial for decarbonizing Building Energy Systems (BES). Despite recent advancements in data-driven Deep Reinforcement Learning (DRL) for BES optimization, significant challenges still exist, such as the time-consuming and data-intensive nature of training DRL controllers and the complexity of environment dynamics in Multi-Agent Reinforcement Learning (MARL). Consequently, these obstacles impede the synchronization and coordination of multiple agent control, leading to slow DRL convergence performance. To address these issues. This paper proposes a novel approach to optimize hybrid building energy systems. We introduce an integrated system combining a multi-stage Proximal Policy Optimization (PPO) on-policy framework with Imitation Learning (IL), interacting with the model environment. To improve scalability and robustness of Multi-agent Systems (MAS), this approach is designed to enhance training efficiency with centralized training and decentralized execution. Simulation results of case studies demonstrate the effectiveness of the Multi-agent Deep Reinforcement Learning (MADRL) model in optimizing the operations of hybrid building energy systems in terms of indoor thermal comfort and energy efficiency. Results show the proposed framework significantly improve performance in achieving convergence in just 50 episodes for dynamic decision-making. The scalability and robustness of the proposed model have been validated across various scenarios. Compared with the baseline during cold and warm weeks, the proposed control approach achieved improvements of 34.86% and 46.10% in energy self-sufficiency ratio, respectively. Additionally, the developed MADRL effectively improved solar photovoltaic (PV) self-consumption and reduced household energy costs. Notably, it increased the average indoor temperature closer to the desired set-point by 1.33 °C, and improved the self-consumption ratio by 15.78% in the colder week and 18.47% in the warmer week, compared to baseline measurements. These findings highlight the advantages of the multi-stage PPO on-policy framework, enabling faster learning and reduced training time, resulting in cost-effective solutions and enhanced solar PV self-consumption.

Keywords: Multi-agent reinforcement learning; Schedule optimization; Multi-stage; Thermal comfort; Energy cost (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (8)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261924007979
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:367:y:2024:i:c:s0306261924007979

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2024.123414

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().