An efficient and privacy-preserving algorithm for multiple energy hubs scheduling with federated and matching deep reinforcement learning

Minghao Chen, Yi Sun, Zhiyuan Xie, Nvgui Lin and Peng Wu

Energy, 2023, vol. 284, issue C

Abstract: As a significant paradigm change in reinforcement learning, federated learning (FL) has emerged to address the efficiency bottlenecks and privacy concerns of centralized training. However, the discrepancy of multi-energy demands among energy hubs (EHs) may cause undesirable instability of FL mechanisms. This study proposes a scheduling algorithm for adaptively controlling multiple EHs with combined heat and power, gas boiler, and electrical boiler. We first present the formulations of twin delayed deep deterministic policy gradient (TD3) to schedule the energy conversion under the time-varying price and demands of electricity, heat, and natural gas. The novelty of the proposed algorithm lies in developing a distributed TD3-agents training that consists of FL and matching-based agent-to-agent learning, namely matching learning (ML). Specifically, the FL is first introduced for collaborative training of EH's TD3 agents by sharing their semi-trained scheduling models. The ML innovatively proposed for addressing the training instability of FL is formulated as a matching game in which the poor-trained agents learn the scheduling knowledge from the agent who has analogous demand patterns and better reward. Simulation results display the advantage and feasibility of the proposed algorithm in terms of the speed of training convergence, energy consumption of equipment, and economic benefit.

Keywords: Energy hub; Federated learning; Energy scheduling; Matching game: twin delayed deep deterministic policy gradient (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544223020352
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:284:y:2023:i:c:s0360544223020352

DOI: 10.1016/j.energy.2023.128641

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().