Proximal Policy Optimization for Energy Management of Electric Vehicles and PV Storage Units

Monica Alonso, Hortensia Amaris (), David Martin and Arturo de la Escalera
Additional contact information
Monica Alonso: Department of Electrical Engineering, Universidad Carlos III de Madrid, 28911 Leganes, Spain
Hortensia Amaris: Department of Electrical Engineering, Universidad Carlos III de Madrid, 28911 Leganes, Spain
David Martin: Department of Electrical Engineering, Universidad Carlos III de Madrid, 28911 Leganes, Spain
Arturo de la Escalera: Department of Electrical Engineering, Universidad Carlos III de Madrid, 28911 Leganes, Spain

Energies, 2023, vol. 16, issue 15, 1-20

Abstract: Connected autonomous electric vehicles (CAEVs) are essential actors in the decarbonization process of the transport sector and a key aspect of home energy management systems (HEMSs) along with PV units, CAEVs and battery energy storage systems. However, there are associated uncertainties which present new challenges to HEMSs, such as aleatory EV arrival and departure times, unknown EV battery states of charge at the connection time, and stochastic PV production due to weather and passing cloud conditions. The proposed HEMS is based on proximal policy optimization (PPO), which is a deep reinforcement learning algorithm suitable for continuous complex environments. The optimal solution for HEMS is a tradeoff between CAEV driver’s range anxiety, batteries degradation, and energy consumption, which is solved by means of incentives/penalties in the reinforcement learning formulation. The proposed PPO algorithm was compared to conventional methods such as business-as-usual (BAU) and value iteration (VI) solutions based on dynamic programming. Simulation results indicate that the proposed PPO’s performance showed a daily energy cost reduction of 54% and 27% compared to BAU and VI, respectively. Finally, the developed PPO algorithm is suitable for real-time operations due to its fast execution and good convergence to the optimal solution.

Keywords: autonomous electric vehicle; energy storage; home energy management; reinforcement learning (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/16/15/5689/pdf (application/pdf)
https://www.mdpi.com/1996-1073/16/15/5689/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:16:y:2023:i:15:p:5689-:d:1205941

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().