Two-Stage Optimization of Virtual Power Plant Operation Considering Substantial Quantity of EVs Participation Using Reinforcement Learning and Gradient-Based Programming

Rong Zhu, Jiwen Qi, Jiatong Wang and Li Li ()
Additional contact information
Rong Zhu: School of Electrical and Data Engineering, University of Technology Sydney, 15 Broadway, Ultimo 2007, Australia
Jiwen Qi: School of Electrical and Data Engineering, University of Technology Sydney, 15 Broadway, Ultimo 2007, Australia
Jiatong Wang: School of Electrical, Computer and Telecommunications Engineering, University of Wollongong, Northfields Avenue, Wollongong 2522, Australia
Li Li: School of Electrical and Data Engineering, University of Technology Sydney, 15 Broadway, Ultimo 2007, Australia

Energies, 2025, vol. 18, issue 22, 1-29

Abstract: Modern electrical vehicles (EVs) are equipped with sizable batteries that possess significant potential as energy prosumers. EVs are poised to be transformative assets and pivotal contributors to the virtual power plant (VPP), enhancing the performance and profitability of VPPs. The number of household EVs is increasing yearly, and this poses new challenges to the optimization of VPP operations. The computational cost increases exponentially as the number of decision variables rises with the increasing participation of EVs. This paper explores the role of a large number of EVs as prosumers, interacting with a VPP consisting of a photovoltaic system and battery energy storage system. To accommodate the large quantity of EVs in the modeling, this research adopts the decentralized control structure. It optimizes EV operations by regulating their charging and discharging behavior in response to pricing signals from the VPP. A two-stage optimization framework is proposed for VPP-EV operation using a reinforcement algorithm and gradient-based programming. Action masking for reinforcement learning is explored to eliminate invalid actions, reducing ineffective exploration, thereby accelerating the convergence of the algorithm. The proposed approach is capable of handling a substantial number of EVs and addressing the stochastic characteristics of EV charging and discharging behaviors. Simulation results demonstrate that the VPP-EV operation optimization increases the revenue of the VPP and significantly reduces the electricity costs for EV owners. Through the optimization of EV operations, the charging cost of 1000 EVs participating in the V2G services is reduced by 26.38% compared to those that opt out of the scheme, and VPP revenue increases by 27.83% accordingly.

Keywords: reinforcement learning; virtual power plant; electrical vehicle (EV); vehicle-to-grid (V2G); gradient-based programming; two-stage optimization (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/18/22/5898/pdf (application/pdf)
https://www.mdpi.com/1996-1073/18/22/5898/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:18:y:2025:i:22:p:5898-:d:1790970

Access Statistics for this article

Energies is currently edited by Ms. Cassie Shen

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().