A constrained DRL-based bi-level coordinated method for large-scale EVs charging

Fangzhu Ming, Feng Gao, Kun Liu and Xingqi Li

Applied Energy, 2023, vol. 331, issue C, No S0306261922016385

Abstract: With the vigorous development of battery electric vehicles (BEVs), BEVs’ charging scheduling is essential for better economy and safety. In this paper, we aim to minimize the electricity purchasing cost considering a large number of BEVs and distributed energy. This problem is challenging to get the optimal charging policy due to a large number of uncertainties and dimension disasters caused by a large scale of BEVs and renewable energy. To meet these challenges, we propose an improved bi-level schedule framework, which decomposes the primal problem into two sub-problems to reduce the computational complexity and designs a communication mechanism to ensure the consistency of optimality between different levels. Then the problem is modeled as constrained multi-level Markov decision processes (CMMDP). In the upper level, a constrained deep reinforcement learning method (CDRL) is proposed to get the total charging or discharging energy of BEV groups. An action constraint module is constructed to ensure the feasibility of chosen actions and a novel reward shaping function is designed to optimize action selection. In the lower level, an optimal descending order charging policy (DOCP) is taken to fast decide the charging or discharging behavior for each BEV based on the upper level’s decision. Numerical experiments show that our method has obvious superiority in training efficiency and solution accuracy compared with state of art DRL methods, and reduces the cost by 12% to 28% compared with an experience charging policy.

Keywords: Stochastic scheduling; Multi-energy system; Hierarchical optimization; Constrained Markov decision processes; Deep reinforcement learning (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261922016385
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:331:y:2023:i:c:s0306261922016385

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2022.120381

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().