A hybrid heuristic-reinforcement learning-based real-time control model for residential behind-the-meter PV-battery systems

Mostafa Rezaeimozafar, Maeve Duffy, Rory F.D. Monaghan and Enda Barrett

Applied Energy, 2024, vol. 355, issue C, No S0306261923016082

Abstract: The behind-the-meter (BTM) energy management problem has recently attracted a lot of attention due to the increase in the number of residential photovoltaic (PV)-battery energy storage systems (BESSs). In this work, the use of deep reinforcement learning (DRL) combined with a novel heuristic model for real-time control of home batteries is investigated. The control problem is formulated as a finite Markov Decision Process with discrete time steps, where a proximal policy optimization (PPO) algorithm is employed to train the DRL agent with discrete action space. The agent is trained using real-world measured data to learn the policy for sequential charge/discharge tasks, aiming to minimize daily electricity costs. The battery power is calculated using an innovative heuristic model considering the agent's decision and the battery's available capacity, ensuring demand-supply balance through PV self-consumption and load demand shifting. The performance of the model is evaluated by comparing it to four RL agents and two benchmark models based on rule-based and scenario-based stochastic optimization strategies. The results confirm that the presented model outperforms its counterparts, offering €80.38 savings on electricity bills over 46 days of the test data set. This figure exceeds the savings of the rule-based and stochastic models by €15.64 and €19.38, respectively.

Keywords: Battery energy storage system; Behind-the-meter systems; Photovoltaic; Proximal policy optimization; Reinforcement learning (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261923016082
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:355:y:2024:i:c:s0306261923016082

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2023.122244

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().