AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (2.2 MB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Q-learning Based Meta-Heuristics for Scheduling Bi-Objective Surgery Problems with Setup Time

School of Computer Science, Liaocheng University, Liaocheng 252000, China
Macau Institute of System Engineering, Macau University of Science and Technology, Macao 999078, China
Department of Electronics and Computer Science, Koszalin University of Technology, Koszalin 75-453, Poland
Show Author Information

Abstract

Since the increasing demand for surgeries in hospitals, the surgery scheduling problems have attracted extensive attention. This study focuses on solving a surgery scheduling problem with setup time. First, a mathematical model is created to minimize the maximum completion time (makespan) of all surgeries and patient waiting time, simultaneously. The time by the fatigue effect is included in the surgery time, which is caused by doctors’ long working time. Second, four mate-heuristics are optimized to address the relevant problems. Three novel strategies are designed to improve the quality of the initial solutions. To improve the convergence of the algorithms, seven local search operators are proposed based on the characteristics of the surgery scheduling problems. Third, Q-learning is used to dynamically choose the optimal local search operator for the current state in each iteration. Finally, by comparing the experimental results of 30 instances, the Q-learning based local search strategy’s effectiveness is verified. Among all the compared algorithms, the improved artificial bee colony (ABC) with Q-learning based local search has the best competitiveness.

References

[1]
E. Demeulemeester, J. Belien, B. Cardoen, and M. Samudra, Operating room planning and scheduling, in Handbook of Healthcare Operations Management: Methods and Applications, pp. 121–152, 2013.
[2]

G. B. Zelenock and C. S. Zambricki, The health care crisis: Impact on surgery in the community hospital setting, Arch. Surg., vol. 136, no. 5, pp. 585–591, 2001.

[3]

F. Guerriero and R. Guido, Operational research in the management of the operating theatre: a survey, Health Care Manag. Sci., vol. 14, no. 1, pp. 89–114, 2011.

[4]

A. Costa, V. Fernandez-Viagas, and J. M. Framinan, Solving the hybrid flow shop scheduling problem with limited human resource constraint, Comput. Ind. Eng., vol. 146, p. 106545, 2020.

[5]

J. H. May, W. E. Spangler, D. P. Strum, and L. G. Vargas, The surgical scheduling problem: Current research and future opportunities, Prod. Oper. Manag., vol. 20, no. 3, pp. 392–405, 2011.

[6]

D. Min and Y. Yih, An elective surgery scheduling problem considering patient priority, Comput. Oper. Res., vol. 37, no. 6, pp. 1091–1099, 2010.

[7]

D. J. Breuer, N. Lahrichi, D. E. Clark, and J. C. Benneyan, Robust combined operating room planning and personnel scheduling under uncertainty, Oper. Res. Health Care, vol. 27, pp. 100276, 2020.

[8]

O. V. Shylo, O. A. Prokopyev, and A. J. Schaefer, Stochastic operating room scheduling for high-volume specialties under block booking, Inf. J. Comput., vol. 25, no. 4, pp. 682–692, 2013.

[9]

I. Ozkarahan, Allocation of surgeries to operating rooms by goal programing, J. Med. Syst., vol. 24, no. 6, pp. 339–378, 2000.

[10]
S. H. Hashemi Doulabi, L. -M. Rousseau, and G. Pesant, A constraint-programming-based branch-and-price-and-cut approach for operating room planning and scheduling, Inf. J. Comput. , vol. 28, no. 3, pp. 432–448, 2016.
[11]

F. Li, D. Gupta, and S. Potthoff, Improving operating room schedules, Health Care Manag. Sci., vol. 19, no. 3, pp. 261–278, 2016.

[12]

B. Cardoen, E. Demeulemeester, and J. Beliën, Optimizing a multiple objective surgical case sequencing problem, Int. J. Prod. Econ., vol. 119, no. 2, pp. 354–366, 2009.

[13]

H. Fei, N. Meskens, and C. Chu, A planning and scheduling problem for an operating theatre using an open scheduling strategy, Comput. Ind. Eng., vol. 58, no. 2, pp. 221–230, 2010.

[14]

S. Lee and Y. Yih, Reducing patient-flow delays in surgical suites through determining start-times of surgical cases, Eur. J. Oper. Res., vol. 238, no. 2, pp. 620–629, 2014.

[15]

P. Demeester, W. Souffriau, P. De Causmaecker, and G. Vanden Berghe, A hybrid tabu search algorithm for automatically assigning patients to beds, Artif. Intell. Med., vol. 48, no. 1, pp. 61–70, 2010.

[16]

W. Xiang, J. Yin, and G. Lim, An ant colony optimization approach for solving an operating room surgery scheduling problem, Comput. Ind. Eng., vol. 85, pp. 335–345, 2015.

[17]

M. Varmazyar, R. Akhavan-Tabatabaei, N. Salmasi, and M. Modarres, Operating room scheduling problem under uncertainty: Application of continuous phase-type distributions, IISE Trans., vol. 52, no. 2, pp. 216–235, 2020.

[18]
Y. -K. Lin and M. -Y. Li, Solving operating room scheduling problem using artificial bee colony algorithm, Healthcare, vol. 9, no. 2, p. 152, 2021.
[19]
C. Rizk and J. -P. Arnaout, ACO for the surgical cases assignment problem, J. Med. Syst. , vol. 36, no. 3, pp. 1891–1899, 2012.
[20]

M. Belkhamsa, B. Jarboui, and M. Masmoudi, Two metaheuristics for solving no-wait operating room surgery scheduling problem under various resource constraints, Comput. Ind. Eng., vol. 126, pp. 494–506, 2018.

[21]

F. Dexter, R. Epstein, R. Traub, Y. Xiao, and D. Warltier, Making management decisions on the day of surgery based on operating room efficiency and patient waiting times, Anesthesiology, vol. 101, no. 6, pp. 1444–1453, 2004.

[22]

J. Wen, N. Geng, and X. Xie, Optimal insertion of customers with waiting time targets, Comput. Oper. Res., vol. 122, pp. 105001, 2020.

[23]

N. Geng and X. Xie, Optimal dynamic outpatient scheduling for a diagnostic facility with two waiting time targets, IEEE Trans. Autom. Contr., vol. 61, no. 12, pp. 3725–3739, 2016.

[24]

X. Pan, N. Geng, X. Xie, and J. Wen, Managing appointments with waiting time targets and random walk-ins, Omega, vol. 95, pp. 102062, 2020.

[25]

J. T. van Essen, E. W. Hans, J. L. Hurink, and A. Oversberg, Minimizing the waiting time for emergency surgery, Oper. Res. Health Care, vol. 1, nos. 2–3, pp. 34–44, 2012.

[26]

Y. Zhou, M. Parlar, V. Verter, and S. Fraser, Surgical scheduling with constrained patient waiting times, Prod. Oper. Manag., vol. 30, no. 9, pp. 3253–3271, 2021.

[27]

B. Addis, G. Carello, A. Grosso, and E. Tànfani, Operating room scheduling and rescheduling: a rolling horizon approach, Flex. Serv. Manuf. J., vol. 28, no. 1, pp. 206–232, 2016.

[28]

Y. Fu, Y. Hou, Z. Wang, X. Wu, K. Gao, and L. Wang, Distributed scheduling problems in intelligent manufacturing systems, Tsinghua Science and Technology, vol. 26, no. 5, pp. 625–645, 2021.

[29]
G. -G. Wang, D. Gao, and W. Pedrycz, Solving multiobjective fuzzy job-shop scheduling problem by a hybrid adaptive differential evolution algorithm, IEEE Trans. Ind. Inform. , vol. 18, no. 12, pp. 8519–8528, 2022.
[30]

X. Guo, M. Zhou, S. Liu, and L. Qi, Multiresource-constrained selective disassembly with maximal profit and minimal energy consumption, IEEE Trans. Autom. Sci. Eng., vol. 18, no. 2, pp. 804–816, 2021.

[31]

D. Lei and B. Su, A multi-class teaching–learning-based optimization for multi-objective distributed hybrid flow shop scheduling, Knowl. Based Syst., vol. 263, pp. 110252, 2023.

[32]

L. Gui, X. Li, Q. Zhang, and L. Gao, Domain knowledge used in meta-heuristic algorithms for the job-shop scheduling problem: Review and analysis, Tsinghua Science and Technology, vol. 29, no. 5, pp. 1368–1389, 2024.

[33]

X. Guo, Z. Zhang, L. Qi, S. Liu, Y. Tang, and Z. Zhao, Stochastic hybrid discrete grey wolf optimizer for multi-objective disassembly sequencing and line balancing planning in disassembling multiple products, IEEE Trans. Autom. Sci. Eng., vol. 19, no. 3, pp. 1744–1756, 2022.

[34]

H. Li, X. Li, and L. Gao, A discrete artificial bee colony algorithm for the distributed heterogeneous no-wait flowshop scheduling problem, Appl. Soft Comput., vol. 100, pp. 106946, 2021.

[35]

Y. Pan, K. Gao, Z. Li, and N. Wu, Improved meta-heuristics for solving distributed lot-streaming permutation flow shop scheduling problems, IEEE Trans. Autom. Sci. Eng., vol. 20, no. 1, pp. 361–371, 2023.

[36]

F. Chen, C. Luo, W. Gong, and C. Lu, Two-stage adaptive memetic algorithm with surprisingly popular mechanism for energy-aware distributed hybrid flow shop scheduling problem with sequence-dependent setup time, Complex System Modeling and Simulation., vol. 4, no. 1, pp. 82–108, 2024.

[37]

D. Lei, Z. Cui, and M. Li, A dynamical artificial bee colony for vehicle routing problem with drones, Eng. Appl. Artif. Intell., vol. 107, p. 2022.

[38]

Y. Fu, Y. Hou, Z. Chen, X. Pu, K. Gao, and A. Sadollah, Modelling and scheduling integration of distributed production and distribution problems via black widow optimization, Swarm Evol. Comput., vol. 68, p. 101015, 2022.

[39]

W. Li, X. Yan, and Y. Huang, Cooperative-guided ant colony optimization with knowledge learning for job shop scheduling problem, Tsinghua Sci. Technol., vol. 29, no. 5, pp. 1283–1299, 2024.

[40]

F. Zhao, D. Shao, L. Wang, T. Xu, N. Zhu, and Jonrinaldi, An effective water wave optimization algorithm with problem-specific knowledge for the distributed assembly blocking flow-shop scheduling problem, Knowl. Based Syst., vol. 243, p. 108471, 2022.

[41]

Y. Fu, M. Zhou, X. Guo, and L. Qi, Scheduling dual-objective stochastic hybrid flow shop with deteriorating jobs via bi-population evolutionary algorithm, IEEE Trans. Syst. Man Cybern. Syst., vol. 50, no. 12, pp. 5037–5048, 2020.

[42]
J. -J. Wang and L. Wang, A bi-population cooperative memetic algorithm for distributed hybrid flow-shop scheduling, IEEE Trans. Emerg. Top. Comput. Intell. , vol. 5, no. 6, pp. 947–961, 2021.
[43]

Y. Wang, Y. Wang, Y. Han, J. Li, K. Gao, and Y. Nojima, Intelligent optimization under multiple factories: Hybrid flow shop scheduling problem with blocking constraints using an advanced iterated greedy algorithm, Complex System Modeling and Simulation, vol. 3, no. 4, pp. 282–306, 2023.

[44]

L. Wang, Z. Pan, and J. Wang, A review of reinforcement learning based intelligent optimization for manufacturing scheduling, Complex System and Modeling and Simulation, vol. 1, no. 4, pp. 257–270, 2021.

[45]
H. Yu, K. -Z. Gao, Z. -F. Ma, and Y. -X. Pan, Improved meta-heuristics with Q-learning for solving distributed assembly permutation flowshop scheduling problems, Swarm Evol. Comput. , vol. 80, p. 101335, 2023.
[46]
X. -L. Chen, J. -Q. Li, and Y. Xu, Q-learning based multi-objective immune algorithm for fuzzy flexible job shop scheduling problem considering dynamic disruptions, Swarm Evol. Comput. , vol. 83, p. 101414, 2023.
[47]
H. Li, K. Gao, P. -Y. Duan, J. -Q. Li, and L. Zhang, An improved artificial bee colony algorithm with Q-learning for solving permutation flow-shop scheduling problems, IEEE Trans. Syst. Man Cybern. Syst. , vol. 53, no. 5, pp. 2684–2693, 2023.
[48]
Z. -Q. Zhang, F. -C. Wu, B. Qian, R. Hu, L. Wang, and H. -P. Jin, A Q-learning-based hyper-heuristic evolutionary algorithm for the distributed flexible job-shop scheduling problem with crane transportation, Expert Syst. Appl. , vol. 234, p. 121050, 2023.
[49]

F. Zhao, S. Di, and L. Wang, A hyperheuristic with Q-learning for the multiobjective energy-efficient distributed blocking flow shop scheduling problem, IEEE Trans. Cybern., vol. 53, no. 5, pp. 3337–3350, 2023.

[50]

M. Gao, K. Gao, Z. Ma, and W. Tang, Ensemble meta-heuristics and Q-learning for solving unmanned surface vessels scheduling problems, Swarm Evol. Comput., vol. 82, pp. 101358, 2023.

[51]
R. Hu, Y. -F. Huang, X. Wu, B. Qian, L. Wang, and Z. -Q. Zhang, Collaborative Q-learning hyper-heuristic evolutionary algorithm for the production and transportation integrated scheduling of silicon electrodes, Swarm Evol. Comput. , vol. 86, p. 101498, 2024.
[52]

F. Zhao, Q. Wang, and L. Wang, An inverse reinforcement learning framework with the Q-learning mechanism for the metaheuristic algorithm, Knowl. Based Syst., vol. 265, p. 2023.

[53]

B. Xi and D. Lei, Q-learning-based teaching-learning optimization for distributed two-stage hybrid flow shop scheduling with fuzzy processing time, Complex System Modeling and Simulation, vol. 2, no. 2, pp. 113–129, 2022.

[54]

H. Yu, K. Gao, N. Wu, M. Zhou, P. N. Suganthan, and S. Wang, Scheduling multiobjective dynamic surgery problems via Q-learning-based meta-heuristics, IEEE Trans. Syst. Man Cybern, Syst., vol. 54, no. 6, pp. 3321–3333, 2024.

[55]
M. -C. Yang, Q. -K. Pan, H. -Y. Sang, W. -M. Li, and Y. -L. Wang, A slack speed-up based discrete artificial bee colony algorithm for resource-constrained operating room scheduling problem, Comput. Ind. Eng. , vol. 186, p. 109760, 2023.
Complex System Modeling and Simulation
Pages 321-338
Cite this article:
Zhang R, Yu H, Slowik A, et al. Q-learning Based Meta-Heuristics for Scheduling Bi-Objective Surgery Problems with Setup Time. Complex System Modeling and Simulation, 2024, 4(4): 321-338. https://doi.org/10.23919/CSMS.2024.0021

124

Views

13

Downloads

0

Crossref

0

Scopus

Altmetrics

Received: 29 May 2024
Revised: 29 July 2024
Accepted: 07 August 2024
Published: 30 December 2024
© The author(s) 2024.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return