Twin-Delayed Deep Deterministic Policy Gradient for Low-Frequency Oscillation Damping Control

Qiushi Cui, Gyoungjae Kim and Yang Weng
Additional contact information
Qiushi Cui: School of Electrical, Computer and Energy Engineering, Arizona State University, 551 East Tyler Mall, Tempe, AZ 85281, USA
Gyoungjae Kim: School of Electrical, Computer and Energy Engineering, Arizona State University, 551 East Tyler Mall, Tempe, AZ 85281, USA
Yang Weng: School of Electrical, Computer and Energy Engineering, Arizona State University, 551 East Tyler Mall, Tempe, AZ 85281, USA

Energies, 2021, vol. 14, issue 20, 1-13

Abstract: Due to the large scale of power systems, latency uncertainty in communications can cause severe problems in wide-area measurement systems. To resolve this issue, a significant amount of past work focuses on using emerging technology, including machine learning methods such as Q-learning, for addressing latency issues in modern controls. Although the method can deal with the stochastic characteristics of communication latency, the Q-values can be overestimated in Q-learning methods, leading to high bias. To address the overestimation bias issue, we redesign the learning structure of the deep deterministic policy gradient (DDPG). Then we develop a damping control twin-delayed deep deterministic policy gradient method to handle the damping control issue under unknown latency in the power network. The purpose is to address the damping control issue under unknown latency in the power network. This paper will create a novel reward algorithm, taking into account the machine speed deviation, the episode termination prevention, and the feedback from action space. In this way, the system optimally damps down frequency oscillations while maintaining the system’s stability and reliable operation within defined limits. The simulation results verify the proposed algorithm in various perspectives, including the latency sensitivity analysis under high renewable energy penetration and the comparison with conventional and machine learning control algorithms. The proposed method shows a fast learning curve and good control performance under varying communication latency.

Keywords: latency; twin-delayed deep deterministic policy gradient; damping control; wide-area measurement systems; low-frequency oscillations (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/14/20/6695/pdf (application/pdf)
https://www.mdpi.com/1996-1073/14/20/6695/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:14:y:2021:i:20:p:6695-:d:656987

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().