Deep reinforcement learning-based two-timescale Volt-VAR control with degradation-aware smart inverters in power distribution systems

Farzana Kabir, Nanpeng Yu, Yuanqi Gao and Wenyu Wang

Applied Energy, 2023, vol. 335, issue C, No S0306261922018864

Abstract: Higher penetration of intermittent solar photovoltaic (PV) systems in the distribution grid results in frequent voltage fluctuations. The conventional voltage regulating devices operating on a slow-timescale need to be supplemented with the fast-operating smart inverters with adjustable reactive power setpoints. Complete and accurate information about distribution network topology and line parameters is necessary for conventional model-based Volt-VAR control (VVC) methods. However, such information is often unavailable. To tackle these challenges, a reinforcement learning-based two-timescale VVC algorithm is proposed in this paper that jointly controls the conventional voltage regulating devices at the slow-timescale and the smart inverters at the fast-timescale. Our proposed VVC algorithm simultaneously minimizes voltage violation costs and system operation costs in a model-free manner utilizing historical operational data. Two hierarchically organized agents are set up for the slow-timescale and fast-timescale problems, which are coupled through a communication scheme. The two sets of control policies are learned concurrently by a deep deterministic policy gradient and multi-agent soft actor-critic algorithm respectively. Comprehensive numerical studies performed with the IEEE 123-bus distribution test feeder show that the proposed framework can identify near optimal control actions of voltage regulating devices and smart inverters in real-time operations.

Keywords: Two-timescale; Volt-VAR control; Smart inverters; High solar PV penetration; Reinforcement learning (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261922018864
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:335:y:2023:i:c:s0306261922018864

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2022.120629

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().