A safe reinforcement learning-based charging strategy for electric vehicles in residential microgrid

Shulei Zhang, Runda Jia, Hengxin Pan and Yankai Cao

Applied Energy, 2023, vol. 348, issue C, No S0306261923008541

Abstract: With the growing popularity of electric vehicles (EVs), it is a new challenge for the residential microgrid system to conduct charging scheduling to meet the charging demands of EVs while maximizing its profit. In this work, a safe reinforcement learning (RL)-based charging scheduling strategy is proposed to meet this challenge. We construct a complete microgrid system equipped with a large charging station and consider different types of EVs, as well as the vehicle-to-grid (V2G) mode and nonlinear charging characteristics of EVs. Subsequently, the charging scheduling problem is formulated as a constrained Markov decision process (CMDP) due to the various limitations of power and demands. To effectively capture the uncertainties of the supply side and demand side of the microgrid, a model-free RL framework is employed. However, the curse of dimensionality of the action space is inevitable as EVs increase. To solve this problem, a charging and discharging strategy based on a general ladder electricity pricing scheme is designed. Different EVs are divided into different sets according to their states under this strategy, and the agent gives control signals of different sets instead of controlling each EV individually, which effectively reduces the dimension of the action space. Subsequently, a constrained soft actor-critic (CSAC) algorithm is designed to solve the established CMDP, and a safety filter is introduced to ensure safety. In the end, a numerical case is conducted to verify the effectiveness of the proposed method.

Keywords: Electric vehicles; Charging scheduling; Safe reinforcement learning; Constrained soft actor-critic; Residential microgrid; Nonlinear charging characteristics (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261923008541
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:348:y:2023:i:c:s0306261923008541

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2023.121490

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().