Robust federated deep reinforcement learning for optimal control in multiple virtual power plants with electric vehicles

Bin Feng, Zhuping Liu, Gang Huang and Chuangxin Guo

Applied Energy, 2023, vol. 349, issue C, No S0306261923009790

Abstract: The deployment of virtual power plants (VPPs) with electric vehicles (EVs) is crucial for the successful integration of renewable energy sources and efficient management of EV charging and discharging while maintaining sustainability and cost-effectiveness. Deep reinforcement learning (DRL) is a highly promising method that uses historical data to learn optimal control strategies and adapts to a wide range of real-time scenarios. To address data privacy concerns in VPPs, federated DRL, which trains models across multiple VPPs, is necessary. However, existing federated DRL methods are prone to disturbance, which can severely impact system performance. This paper proposes a robust federated DRL method to ensure the robustness and reliability of VPP control strategies. Firstly, we formulate the control strategies of multiple VPPs as a Markov decision process that takes into account disturbances, aiming to achieve self-balance as much as possible. Secondly, we employ the stochastically controlled stochastic gradient method to increase training speed. Additionally, we introduce the robust gradient filter to develop a robust federated DRL method based on policy-based DRL. Finally, we validate the effectiveness and robustness of the proposed robust federated DRL method, which maintains balance in internal VPP power.

Keywords: Virtual power plants; Electric vehicles; Federated learning; Deep reinforcement learning (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261923009790
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:349:y:2023:i:c:s0306261923009790

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2023.121615

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().