Resilience enhancement of multi-agent reinforcement learning-based demand response against adversarial attacks

Lanting Zeng, Dawei Qiu and Mingyang Sun

Applied Energy, 2022, vol. 324, issue C, No S0306261922009850

Abstract: Demand response improves grid security by adjusting the flexibility of consumers meanwhile maintaining their demand–supply balance in real-time. With the large-scale deployment of distributed digital communication technologies and advanced metering infrastructures, data-driven approaches such as multi-agent reinforcement learning (MARL) are being widely employed to solve demand response problems. Nevertheless, the massive interaction of data inside and outside the demand response management system may lead to severe threats from the perspective of cyber-attacks. The cyber security requirements of MARL-based demand response problems are less discussed in the existing studies. To this end, this paper proposes a robust adversarial multi-agent reinforcement learning framework for demand response (RAMARL-DR) with an enhanced resilience against adversarial attacks. In particular, the proposed RAMARL-DR first constructs an adversary agent that aims to cause the worst-case performance via formulating an adversarial attack; and then adopts periodic alternating robust adversarial training scenarios with the optimal adversary aiming to diminish the severe impacts induced by adversarial attacks. Case studies are conducted based on an OpenAI Gym environment CityLearn, which provides a standard evaluation platform of MARL algorithms for demand response problems. Empirical results indicate that the MARL-based demand response management system is vulnerable when the adversary agent occurs, and its performance can be significantly improved after periodic alternating robust adversarial training. It can be found that the adversary agent can result in a 41.43% higher metric value of Ramping than the no adversary case, whereas the proposed RAMARL-DR can significantly enhance the system resilience with an approximately 38.85% reduction in the ramping of net demand.

Keywords: Demand response; Cyber security; Resilience; Multi-agent reinforcement learning; Adversarial attacks (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261922009850
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:324:y:2022:i:c:s0306261922009850

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2022.119688

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().