Federated deep reinforcement learning for dynamic job scheduling in cloud-edge collaborative manufacturing systems

Xiaohan Wang, Lin Zhang, Lihui Wang, Xi Vincent Wang and Yongkui Liu

International Journal of Production Research, 2024, vol. 62, issue 21, 7743-7762

Abstract: The cloud-edge collaborative manufacturing system (CCMS) connects distributed factories to a cloud centre through cloud-edge collaborative communication, introducing both opportunities and challenges to conventional dynamic job scheduling. Enhancing each factory's scheduling performance by sharing general scheduling knowledge among heterogeneous factories under the consideration of data privacy protection remains challenging. To this end, this paper proposes to solve the dynamic job scheduling in the context of CCMS with a novel federated deep reinforcement learning (FDRL) approach. Within each factory, the scheduling objective involves minimising the makespan and energy consumption, accounting for machine warm-up procedures and real-time dynamics. To handle heterogeneous policy structures, we aggregate their hidden parameters through FDRL, with states, actions, and rewards designed to facilitate the aggregation. The two-phase algorithm, comprising iterative local training and global aggregation, trains the scheduling policies. Constraint items are introduced to the loss functions to smooth local training, and the global aggregation considers production scales and obtained objectives. The proposed approach enhances the solution quality and generalisation of each factory's scheduling policy without exposing original production data. Numerical experiments conducted on sixty scheduling instances validate the superiority of the proposed approach compared to twelve dynamic scheduling methods. Compared to independently trained DRL-based approaches, the proposed FDRL-based approach achieves up to an 8.9% reduction in makespan and a 22.3% decrease in energy consumption through knowledge sharing.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/00207543.2024.2328116 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tprsxx:v:62:y:2024:i:21:p:7743-7762

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/TPRS20

DOI: 10.1080/00207543.2024.2328116

Access Statistics for this article

International Journal of Production Research is currently edited by Professor A. Dolgui

More articles in International Journal of Production Research from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().