A transformer-based deep reinforcement learning approach for dynamic parallel machine scheduling problem with family setups

Funing Li (), Sebastian Lang (), Yuan Tian (), Bingyuan Hong (), Benjamin Rolf (), Ruben Noortwyck (), Robert Schulz () and Tobias Reggelin ()
Additional contact information
Funing Li: University of Stuttgart
Sebastian Lang: Otto von Guericke University Magdeburg
Yuan Tian: ETH Zürich
Bingyuan Hong: Zhejiang Ocean University
Benjamin Rolf: Otto von Guericke University Magdeburg
Ruben Noortwyck: University of Stuttgart
Robert Schulz: University of Stuttgart
Tobias Reggelin: Otto von Guericke University Magdeburg

Journal of Intelligent Manufacturing, 2025, vol. 36, issue 7, No 14, 4735-4768

Abstract: Abstract The parallel machine scheduling problem (PMSP) involves the optimized assignment of a set of jobs to a collection of parallel machines, which is a proper formulation for the modern manufacturing environment. Deep reinforcement learning (DRL) has been widely employed to solve PMSP. However, the majority of existing DRL-based frameworks still suffer from generalizability and scalability. More specifically, the state and action design still heavily rely on human efforts. To bridge these gaps, we propose a practical reinforcement learning-based framework to tackle a PMSP with new job arrivals and family setup constraints. We design a variable-length state matrix containing full job and machine information. This enables the DRL agent to autonomously extract features from raw data and make decisions with a global perspective. To efficiently process this novel state matrix, we elaborately modify a Transformer model to represent the DRL agent. By integrating the modified Transformer model to represent the DRL agent, a novel state representation can be effectively leveraged. This innovative DRL framework offers a high-quality and robust solution that significantly reduces the reliance on manual effort traditionally required in scheduling tasks. In the numerical experiment, the stability of the proposed agent during training is first demonstrated. Then we compare this trained agent on 192 instances with several existing approaches, namely a DRL-based approach, a metaheuristic algorithm, and a dispatching rule. The extensive experimental results demonstrate the scalability of our approach and its effectiveness across a variety of scheduling scenarios. Conclusively, our approach can thus solve the scheduling problems with high efficiency and flexibility, paving the way for application of DRL in solving complex and dynamic scheduling problems.

Keywords: Deep reinforcement learning; Dynamic parallel machine scheduling; New job arrivals; Family setups; Multi-head attention (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10845-024-02470-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joinma:v:36:y:2025:i:7:d:10.1007_s10845-024-02470-8

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10845

DOI: 10.1007/s10845-024-02470-8

Access Statistics for this article

Journal of Intelligent Manufacturing is currently edited by Andrew Kusiak

More articles in Journal of Intelligent Manufacturing from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().