Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates

Siebe Paesschesoone, Nezmin Kayedpour, Carlo Manna and Guillaume Crevecoeur

Applied Energy, 2024, vol. 368, issue C, No S0306261924008900

Abstract: This paper presents a novel data-driven approach that leverages reinforcement learning to enhance the efficiency and safety of existing energy flexibility controllers, addressing challenges posed by the dynamic and uncertain nature of modern energy landscapes. With the increasing integration of renewable energy sources, conventional controllers struggle to maintain both safety and optimality. Our proposed approach introduces two significant contributions to standard RL approaches: a data-driven predictive safety filter and an online changepoint detection and policy updating module. Through continuous constraint satisfaction, the predictive safety filter guarantees absolute safety of the proposed controller. Meanwhile, the changepoint detection and policy updating module, inspired by the concept of continual learning, enhances the controller’s adaptivity to non-stationary environments. By identifying changes in the environment, it triggers relearning of the agent, making the controller resilient to evolving conditions. Validation of our approach is conducted on a grid-connected PV-battery-load system, demonstrating its effectiveness in simultaneously improving safety and performance over traditional learning methods. More specifically, the proposed solution was able to increase the energy flexibility by reducing energy costs with 9.3%.

Keywords: Energy flexibility control; Safe continual reinforcement learning; Predictive safety filter; Changepoint detection; Policy updating (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261924008900
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:368:y:2024:i:c:s0306261924008900

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2024.123507

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().