System Frequency Control Method Driven by Deep Reinforcement Learning and Customer Satisfaction for Thermostatically Controlled Load

Rusi Chen, Haiguang Liu, Chengquan Liu (), Guangzheng Yu, Xuan Yang and Yue Zhou
Additional contact information
Rusi Chen: State Grid Hubei Electric Power Research Institute, Wuhan 430077, China
Haiguang Liu: State Grid Hubei Electric Power Research Institute, Wuhan 430077, China
Chengquan Liu: Department of Electrical Engineering, Shanghai University of Electric Power, Shanghai 201306, China
Guangzheng Yu: Department of Electrical Engineering, Shanghai University of Electric Power, Shanghai 201306, China
Xuan Yang: State Grid Hubei Electric Power Co., Ltd., Wuhan 430072, China
Yue Zhou: State Grid Hubei Electric Power Co., Ltd., Wuhan 430072, China

Energies, 2022, vol. 15, issue 21, 1-19

Abstract: The intermittence and fluctuation of renewable energy aggravate the power fluctuation of the power grid and pose a severe challenge to the frequency stability of the power system. Thermostatically controlled loads can participate in the frequency regulation of the power grid due to their flexibility. Aiming to solve the problem of the traditional control methods, which have limited adjustment ability, and to have a positive influence on customers, a deep reinforcement learning control strategy based on the framework of soft actor–critic is proposed, considering customer satisfaction. Firstly, the energy storage index and the discomfort index of different users are defined. Secondly, the fuzzy comprehensive evaluation method is applied to evaluate customer satisfaction. Then, the multi-agent models of thermostatically controlled loads are established based on the soft actor–critic algorithm. The models are trained by using the local information of thermostatically controlled loads, and the comprehensive evaluation index fed back by users and the frequency deviation. After training, each agent can realize the cooperative response of thermostatically controlled loads to the system frequency only by relying on the local information. The simulation results show that the proposed strategy can not only reduce the frequency fluctuation, but also improve customer satisfaction.

Keywords: thermostatically controlled load; frequency regulation; customer satisfaction; soft actor–critic; energy storage index; discomfort index (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/15/21/7866/pdf (application/pdf)
https://www.mdpi.com/1996-1073/15/21/7866/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:15:y:2022:i:21:p:7866-:d:951269

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().