Improving real-time energy decision-making model with an actor-critic agent in modern microgrids with energy storage devices

Karim Bio Gassi and Mustafa Baysal

Energy, 2023, vol. 263, issue PE

Abstract: The hereby study combines a reinforcement learning machine and a myopic optimization model to improve the real-time energy decisions in microgrids with renewable sources and energy storage devices. The reinforcement learning-based agent is built as an actor-critic agent making the aggregated near-optimal charging/discharging energy decisions of the microgrid energy storage devices from a discrete action space relying on a reward related to the microgrid online optimal objective function value. The next step time energy levels of storage devices are then computed and provided to the myopic optimization-based decision-making model as parameters which optimally find the incurred power flow within the microgrid minimizing the real-time microgrid energy cost. The real-time measurement of stochastic parameters of the microgrid coupled with the current energy levels of electrical and heat storage are input to the artificially intelligent machine as observations states. The actor-critic agent approximators are modeled as deep neural networks optimized using the Adam gradient descent algorithm with a gradient threshold. Although the proposed model with a 2-kWh increment of the charging/discharging energy training is time-consuming, it has been able at 100% to optimally make microgrid energy decisions and improve online energy decisions by 90.98% compared to the myopic model alone.

Keywords: Actor-critic; Reinforcement learning; Linear programming; Microgrid; Energy management system; Power flow (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544222029917
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:263:y:2023:i:pe:s0360544222029917

DOI: 10.1016/j.energy.2022.126105

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().