Active Exploration by Chance-Constrained Optimization for Voltage Regulation with Reinforcement Learning

Zhenhuan Ding, Xiaoge Huang and Zhao Liu
Additional contact information
Zhenhuan Ding: Department of Electrical and Computer Engineering, State University of New York at Binghamton, New York, NY 13902, USA
Xiaoge Huang: Department of Electrical and Computer Engineering, State University of New York at Binghamton, New York, NY 13902, USA
Zhao Liu: School of Electrical Engineering, Beijing Jiaotong University, Beijing 100044, China

Energies, 2022, vol. 15, issue 2, 1-17

Abstract: Voltage regulation in distribution networks encounters a challenge of handling uncertainties caused by the high penetration of photovoltaics (PV). This research proposes an active exploration (AE) method based on reinforcement learning (RL) to respond to the uncertainties by regulating the voltage of a distribution network with battery energy storage systems (BESS). The proposed method integrates engineering knowledge to accelerate the training process of RL. The engineering knowledge is the chance-constrained optimization. We formulate the problem in a chance-constrained optimization with a linear load flow approximation. The optimization results are used to guide the action selection of the exploration for improving training efficiency and reducing the conserveness characteristic. The comparison of methods focuses on how BESSs are used, training efficiency, and robustness under varying uncertainties and BESS sizes. We implement the proposed algorithm, a chance-constrained optimization, and a traditional Q-learning in the IEEE 13 Node Test Feeder. Our evaluation shows that the proposed AE method has a better response to the training efficiency compared to traditional Q-learning. Meanwhile, the proposed method has advantages in BESS usage in conserveness compared to the chance-constrained optimization.

Keywords: voltage regulation; chance-constrained optimization; reinforcement learning; uncertainties; battery; active exploration (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/1996-1073/15/2/614/pdf (application/pdf)
https://www.mdpi.com/1996-1073/15/2/614/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:15:y:2022:i:2:p:614-:d:725819

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().