AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Complex System Modeling and Simulation Article
PDF (6.3 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Optimal Design of Flexible Job Shop Scheduling Under Resource Preemption Based on Deep Reinforcement Learning

School of Automation Science and Eletrical Engineering, Beihang University, Beijing 100191, China
Show Author Information

Abstract

With the popularization of multi-variety and small-batch production patterns, the flexible job shop scheduling problem (FJSSP) has been widely studied. The sharing of processing resources by multiple machines frequently occurs due to space constraints in a flexible shop, which results in resource preemption for processing workpieces. Resource preemption complicates the constraints of scheduling problems that are otherwise difficult to solve. In this paper, the flexible job shop scheduling problem under the process resource preemption scenario is modeled, and a two-layer rule scheduling algorithm based on deep reinforcement learning is proposed to achieve the goal of minimum scheduling time. The simulation experiments compare our scheduling algorithm with two traditional metaheuristic optimization algorithms among different processing resource distribution scenarios in static scheduling environment. The results suggest that the two-layer rule scheduling algorithm based on deep reinforcement learning is more effective than the meta-heuristic algorithm in the application of processing resource preemption scenarios. Ablation experiments, generalization, and dynamic experiments are performed to demonstrate the excellent performance of our method for FJSSP under resource preemption.

Keywords

deep reinforcement learningflexible job shop schedulingresource preemptiontwo-level scheduling

References

1
L. Zhang, Research on flexible job shop scheduling based on multi-objective optimization algorithm, MA dissertation, College of Information Science and Engineering, Shandong Agricultural University, Taian, Shangdong, 2019.
2

X. Zhu, Flexible job shop scheduling multi-object optimization based on strong reproduction NSGA-Ⅱ algorithm, Modular Machine Tool and Automatic Manufacturing Technology, no. 9, pp. 180–184, 2021.
Google Scholar
3

Y. Yuan and H. Xu, Flexible job shop scheduling using hybrid differential evolution algorithms, Computers&Industrial Engineering, vol. 65, no. 2, pp. 246–260, 2013.
Crossref Google Scholar
4

Q. Yu, L. Zhao, and S. Pan, A scheduling optimization of flexible job-shop using genetic algorithm, Modular Machine Tool and Automatic Manufacturing Technology, no. 4, pp. 32–34, 2004.
Google Scholar
5

H. Luo and D. Pan, Two-layer coding discrete cuckoo algorithm for solving flexible workshop scheduling problem, Computer and Digital Engineering, vol. 49, no. 7, pp. 1281–1285, 2021.
Google Scholar
6

W. Teekeng, A. Thammano, P. Unkaw, and J. Kiatwuthiamorn, A new algorithm for flexible job shop scheduling problem based on particle swarm optimization, Artificial Life and Robotics, vol. 21, pp. 18–23, 2016.
Crossref Google Scholar
7

V. Boyer, J. Vallikavungal, X. C. Rodríguez, and M. A. Salazar-Aguilar, The generalized flexible job shop scheduling problem, Computers&Industrial Engineering, vol. 160, p. 107542, 2021.
Crossref Google Scholar
8

W. Liu, Y. Gong, W. Chen, Z. Liu, H. Wang, and J. Zhang, Coordinated charging scheduling of electric vehicles: A mixed-variable differential evolution approach, IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 12, pp. 5094–5109, 2019.
Crossref Google Scholar
9

S. Zhou, L. Xing, X. Zheng, N. Du, L. Wang, and Q. Zhang, A self-adaptive differential evolution algorithm for scheduling a single batch-processing machine with arbitrary job sizes and release times, IEEE Transactions on Cybernetics, vol. 51, no. 3, pp. 1430–1442, 2019.
Crossref Google Scholar
10
B. C. Csáji, M. László, and B. Kádár, Reinforcement learning in a distributed market-based production control system, Advanced Engineering Informatics, vol. 20, no. 3, pp. 279–288, 2006.https://doi.org/10.1016/j.aei.2006.01.001
Crossref
11

F. Zhao, X. He, and L. Wang, A two-stage cooperative evolutionary algorithm with problem-specific knowledge for energy-efficient scheduling of no-wait flow-shop problem, IEEE Transactions on Cybernetics, vol. 51, no. 11, pp. 5291–5303, 2020.
Crossref Google Scholar
12

F. Zhao, L. Zhao, L. Wang, and H. Song, An ensemble discrete differential evolution for the distributed blocking flowshop scheduling with minimizing makespan criterion, Expert Systems with Applications, vol. 160, p. 113678, 2020.
Crossref Google Scholar
13

L. Wang, Z. Pan, and J. Wang, A review of reinforcement learning based intelligent optimization for manufacturing scheduling, Complex System Modeling and Simulation, vol. 1, no. 4, pp. 257–270, 2021.
Crossref Google Scholar
14
T. Gabel and M. Riedmiller, Scaling adaptive agent-based reactive job-shop scheduling to large-scale problems, in Proc. 2007 IEEE Symposium on Computational Intelligence in Scheduling, Honolulu, HI, USA, 2007, pp. 259–266.https://doi.org/10.1109/SCIS.2007.367699
Crossref
15
Y. Martínez, A. Nowé, J. Suárez, and R. Bello, A reinforcement learning approach for the flexible job shop scheduling problem, in Proc. 5th International Conference on Learning and Intelligent Optimization, Rome, Italy, 2011, pp. 253–262.https://doi.org/10.1007/978-3-642-25566-3_19
Crossref
16

R. Chen, B. Yang, S. Li, and S. Wang, A self-learning genetic algorithm based on reinforcement learning for flexible job shop scheduling problem, Computers and Industrial Engineering, vol. 149, p. 106778, 2020.
Crossref Google Scholar
17

S. Luo, Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning, Applied Soft Computing, vol. 91, p. 106208, 2020.
Crossref Google Scholar
18

X. Wang, L. Zhang, T. Lin, C. Zhao, K. Wang, and Z. Chen, Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning, Robotics and Computer-Integrated Manufacturing, vol. 77, p. 102324, 2022.
Crossref Google Scholar
19
C. Zhang, W. Song, Z. Cao, J. Zhang, P. S. Tan, and C. Xu, Learning to dispatch for job shop scheduling via deep reinforcement learning, arXiv preprint arXiv: 2010.12367, 2020.
20

I. A. Chaudhry and A. A. Khan, A research survey: Review of flexible job shop scheduling techniques, International Transactions in Operational Research, vol. 23, no. 3, pp. 551–591, 2016.
Crossref Google Scholar
21

J. Peng, M. Liu, M. Zhang, and X. Zhang, Review on Scheduling Algorithms for MOFJSP, China Mechanical Engineering, vol. 23, pp. 3244–3254, 2014.
Google Scholar
22
M. V. Otterlo and M. Wiering, Reinforcement learning and markov decision processes, in Reinforcement Learning, M. Wiering and M. V. Otterlo, eds. Berlin, Germany: Springer-Verlarg, 2012, pp. 3–42.https://doi.org/10.1007/978-3-642-27645-3_1
Crossref
23
L. Graesser and W. L. Keng, Foundations of Deep Reinforcement Learning: Theory and Practice in Python. Boston, MA, USA: Addison-Wesley Professional, 2019.
24

M. Cui, J. Wang, and M. Yue, Machine learning-based anomaly detection for load forecasting under cyberattacks, IEEE Transactions on Smart Grid, vol. 10, no. 5, pp. 5724–5734, 2019.
Crossref Google Scholar
25
T. Zhao and M. Eskenazi, Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning, arXiv preprint arXiv: 1606.02560, 2016.https://doi.org/10.18653/v1/W16-3601
Crossref
26
N. Wei, A Guide to the Implementation of Deep Reinforcement Learning (in Chinese). Beijing, China: Publishing House of Electronics Industry, 2021.
27
Y. Wang, H. He, and X. Tan, Truly proximal policy optimization, in Proc. 35th Uncertainty in Artificial Intelligence, Tel Aviv, Israel, 2019, pp. 113–122.
28
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv: 1707.06347, 2017.
Complex System Modeling and Simulation
Volume 2 Issue 2,
June 2022
Pages 174-185
DOI: 10.23919/CSMS.2022.0007
Cite this article:
Chen Z, Zhang L, Wang X, et al. Optimal Design of Flexible Job Shop Scheduling Under Resource Preemption Based on Deep Reinforcement Learning. Complex System Modeling and Simulation, 2022, 2(2): 174-185. https://doi.org/10.23919/CSMS.2022.0007

918

Views

69

Downloads

10

Crossref

11

Scopus

Altmetrics
Received: 04 March 2022
Revised: 30 March 2022
Accepted: 06 May 2022
Published: 30 June 2022
© The author(s) 2022

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return