Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning

Yujian Ye, Dawei Qiu, Huiyu Wang, Yi Tang and Goran Strbac
Additional contact information
Yujian Ye: School of Electrical Engineering, Southeast University, Nanjing 210096, China
Dawei Qiu: Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK
Huiyu Wang: School of Electrical Engineering, Southeast University, Nanjing 210096, China
Yi Tang: School of Electrical Engineering, Southeast University, Nanjing 210096, China
Goran Strbac: Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK

Energies, 2021, vol. 14, issue 3, 1-22

Abstract: With the roll-out of smart meters and the increasing prevalence of distributed energy resources (DERs) at the residential level, end-users rely on home energy management systems (HEMSs) that can harness real-time data and employ artificial intelligence techniques to optimally manage the operation of different DERs, which are targeted toward minimizing the end-user’s energy bill. In this respect, the performance of the conventional model-based demand response (DR) management approach may deteriorate due to the inaccuracy of the employed DER operating models and the probabilistic modeling of uncertain parameters. To overcome the above drawbacks, this paper develops a novel real-time DR management strategy for a residential household based on the twin delayed deep deterministic policy gradient (TD3) learning approach. This approach is model-free, and thus does not rely on knowledge of the distribution of uncertainties or the operating models and parameters of the DERs. It also enables learning of neural-network-based and fine-grained DR management policies in a multi-dimensional action space by exploiting high-dimensional sensory data that encapsulate the uncertainties associated with the renewable generation, appliances’ operating states, utility prices, and outdoor temperature. The proposed method is applied to the energy management problem for a household with a portfolio of the most prominent types of DERs. Case studies involving a real-world scenario are used to validate the superior performance of the proposed method in reducing the household’s energy costs while coping with the multi-source uncertainties through comprehensive comparisons with the state-of-the-art deep reinforcement learning (DRL) methods.

Keywords: demand response; distributed energy resources; deep neural network; deep reinforcement learning; renewable energy; smart grid (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
https://www.mdpi.com/1996-1073/14/3/531/pdf (application/pdf)
https://www.mdpi.com/1996-1073/14/3/531/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:14:y:2021:i:3:p:531-:d:484004

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().