Physics-model-free heat-electricity energy management of multiple microgrids based on surrogate model-enabled multi-agent deep reinforcement learning

Sichen Li, Weihao Hu, Di Cao, Zhe Chen, Qi Huang, Frede Blaabjerg and Kaiji Liao

Applied Energy, 2023, vol. 346, issue C, No S0306261923007237

Abstract: Reliability and cost-effectiveness in the operation of the multiple microgrid (MMG) system depend on the skillful management of its energy resources. Traditional energy management approaches are physics-model-based, which rely on the precise system parameters (e.g., line parameters) of the electricity and heat network. It is difficult to establish the precise system parameters in practice because they depend on a variety of factors. In this context, this paper proposes a physics-model-free control framework for the energy management of MMGs with heat-electricity energy, consisting of proposed surrogate model and multi-agent deep reinforcement learning (MADRL) approach. An important step is to use historical data to train a surrogate model in supervised manner that can imitate the realistic power and thermal flow calculations. Meanwhile, the energy management problem is reformulated as a Markov game. It is solved by the proposed MADRL-based approach by modeling each MG controller as an agent with a specific objective. The historical trajectories representation, parameter space technology, and deep dense architecture in reinforcement learning are introduced in MADRL to overcome the negative impact brought by the time series data from the input state on the decision-making process and construct an efficient exploration mechanism to overcome inefficient optimization of the MMG system in a multi-agent setting. During training period of MADRL, the trained surrogate models integrate into the environment of the MADRL, which can develop optimal energy management strategy based on the continuous interaction with the surrogate models. The proposed surrogate model enabled MADRL approach can reduce the reliance on precise physical systems and prevent having an impact on the real system while being trained involving trial and error process. Simulation results demonstrate the effectiveness of the proposed control framework.

Keywords: Energy management of multiple microgrids; Combined heat and power; Sparse variational Gaussian processes; Multi-agent deep reinforcement learning (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261923007237
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:346:y:2023:i:c:s0306261923007237

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2023.121359

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().