AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (3.9 MB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

A Q-Learning Based Hybrid Meta-Heuristic for Integrated Scheduling of Disassembly and Reprocessing Processes Considering Product Structures and Stochasticity

School of Business, Qingdao University, Qingdao 266071, China
Macau Institute of Systems Engineering, Macau University of Science and Technology, Macao 999078, China
Department of Industrial Engineering & Innovation Sciences, Eindhoven University of Technology, Eindhoven, 5600 MB, the Netherlands
College of Information Science and Engineering, Northeastern University, Shenyang 110819, China
Show Author Information

Abstract

Remanufacturing is regarded as a sustainable manufacturing paradigm of energy conservation and environment protection. To improve the efficiency of the remanufacturing process, this work investigates an integrated scheduling problem for disassembly and reprocessing in a remanufacturing process, where product structures and uncertainty are taken into account. First, a stochastic programming model is developed to minimize the maximum completion time (makespan). Second, a Q-learning based hybrid meta-heuristic (Q-HMH) is specially devised. In each iteration, a Q-learning method is employed to adaptively choose a premium algorithm from four candidate ones, including genetic algorithm (GA), artificial bee colony (ABC), shuffled frog-leaping algorithm (SFLA), and simulated annealing (SA) methods. At last, simulation experiments are carried out by using sixteen instances with different scales, and three state-of-the-art algorithms in literature and an exact solver CPLEX are chosen for comparisons. By analyzing the results with the average relative percentage deviation (RPD) metric, we find that Q-HMH outperforms its rivals by 9.79%−26.76%. The results and comparisons verify the excellent competitiveness of Q-HMH for solving the concerned problems.

References

[1]
J. Chen, H. Wang, and Y. Fu, A multi-stage supply chain disruption mitigation strategy considering product life cycle during COVID-19, Environ. Sci. Pollut. Res.
[2]

K. Li, L. Zhang, H. Fu, and B. Liu, The effect of intelligent manufacturing on remanufacturing decisions, Comput. Ind. Eng., vol. 178, p. 109114, 2023.

[3]

P. Liang, Y. Fu, and K. Gao, Multi-product disassembly line balancing optimization method for high disassembly profit and low energy consumption with noise pollution constraints, Eng. Appl. Artif. Intell., vol. 130, p. 107721, 2024.

[4]

C. Liu, X. Meng, C. Liu, and Z. Liu, Carbon footprint-based optimization method for remanufacturing machining paths, Int. J. Adv. Manuf. Technol., vol. 124, no. 10, pp. 3391–3406, 2023.

[5]
W. Wang, G. Tian, H. Zhang, K. Xu, and Z. Miao, Modeling and scheduling for remanufacturing systems with disassembly, reprocessing, and reassembly considering total energy consumption, Environ. Sci. Pollut. Res. Int.
[6]

W. Zhang, J. Shi, S. Zhang, and M. Chen, Scenario-based robust remanufacturing scheduling problem using improved biogeography-based optimization algorithm, IEEE Trans. Syst. Man Cybern. Syst., vol. 53, no. 6, pp. 3414–3427, 2023.

[7]

Y. Fu, G. Tian, A. M. Fathollahi-Fard, A. Ahmadi, and C. Zhang, Stochastic multi-objective modelling and optimization of an energy-conscious distributed permutation flow shop scheduling problem with the total tardiness constraint, J. Clean. Prod., vol. 226, pp. 515–525, 2019.

[8]

J. Li, X. Li, and L. Gao, An operator-inspired framework for metaheuristics and its applications on job-shop scheduling problems, Appl. Soft Comput., vol. 157, p. 111522, 2024.

[9]

Y. P. Fu, G. D. Tian, Z. W. Li, and Z. L. Wang, Parallel machine scheduling with dynamic resource allocation via a master–slave genetic algorithm, IEEJ Transactions on Electrical and Electronic Engineering, vol. 13, no. 5, pp. 748–756, 2018.

[10]
G. Tian, X. Zhang, A. M. Fathollahi-Fard, Z. Jiang, C. Zhang, G. Yuan, and D. T. Pham, Hybrid evolutionary algorithm for stochastic multiobjective disassembly line balancing problem in remanufacturing, Environ. Sci. Pollut. Res. Int.
[11]

J. M. Yu, J. S. Kim, and D. H. Lee, Scheduling algorithms to minimise the total family flow time for job shops with job families, Int. J. Prod. Res., vol. 49, no. 22, pp. 6885–6903, 2011.

[12]

Y. Xiao, J. Zhou, S. Xing, and X. Zhu, Research on assembly sequence optimization classification method of remanufacturing parts based on different precision levels, Processes, vol. 11, no. 2, p. 383, 2023.

[13]

H. H. Doh and D. H. Lee, Integrated disassembly and reprocessing lot-sizing for multi-level structured products in remanufacturing systems, Eng. Optim., vol. 54, no. 9, pp. 1476–1493, 2022.

[14]

Y. Fu, M. Zhou, X. Guo, and L. Qi, Stochastic multi-objective integrated disassembly-reprocessing-reassembly scheduling via fruit fly optimization algorithm, J. Clean. Prod., vol. 278, p. 123364, 2021.

[15]

W. Zhang, Y. Zheng, and R. Ahmad, An energy-efficient multi-objective integrated process planning and scheduling for a flexible job-shop-type remanufacturing system, Adv. Eng. Inform., vol. 56, p. 102010, 2023.

[16]

X. Li, N. Li, I. Kolmanovsky, and B. I. Epureanu, Stochastic model predictive control for remanufacturing system management, J. Manuf. Syst., vol. 59, pp. 355–366, 2021.

[17]
G. Yuan, Y. Yang, G. Tian, and A. M. Fathollahi-Fard, Capacitated multi-objective disassembly scheduling with fuzzy processing time via a fruit fly optimization algorithm, Environ. Sci. Pollut. Res.
[18]
Y. Fu, M. Zhou, X. Guo, L. Qi, K. Gao, and A. Albeshri, Multiobjective scheduling of energy-efficient stochastic hybrid open shop with brain storm optimization and simulation evaluation, IEEE Trans. Syst. Man Cybern. Syst.
[19]

Y. Lv, C. Li, X. Zhao, L. Li, and J. Li, A novel approach for remanufacturing process planning considering uncertain and fuzzy information, Front. Mech. Eng., vol. 16, no. 3, pp. 546–558, 2021.

[20]

W. K. Zhang, Y. F. Zheng, and R. Ahmad, An energy-efficient multi-objective scheduling for flexible job-shop-type remanufacturing system, J. Manuf. Syst., vol. 66, pp. 211–232, 2023.

[21]
P. Liang, Y. Fu, S. Ni, and B. Zheng, Modeling and optimization for noise-aversion and energy-awareness disassembly sequence planning problems in reverse supply chain, Environ. Sci. Pollut. Res. Int.
[22]

Y. Fu, M. Zhou, X. Guo, L. Qi, and K. Sedraoui, Multiverse optimization algorithm for stochastic biobjective disassembly sequence planning subject to operation failures, IEEE Trans. Syst. Man Cybern. Syst., vol. 52, no. 2, pp. 1041–1051, 2022.

[23]

D. Yu, X. Zhang, G. Tian, Z. Jiang, Z. Liu, T. Qiang, and C. Zhan, Disassembly sequence planning for green remanufacturing using an improved whale optimisation algorithm, Processes, vol. 10, no. 10, p. 1998, 2022.

[24]

Y. Gao, S. Lou, H. Zheng, and J. Tan, A data-driven method of selective disassembly planning at end-of-life under uncertainty, J. Intell. Manuf., vol. 34, no. 2, pp. 565–585, 2023.

[25]

X. Guo, M. Zhou, S. Liu, and L. Qi, Multiresource-constrained selective disassembly with maximal profit and minimal energy consumption, IEEE Trans. Autom. Sci. Eng., vol. 18, no. 2, pp. 804–816, 2021.

[26]

C. Zhang, A. M. Fathollahi-Fard, J. Li, G. Tian, and T. Zhang, Disassembly sequence planning for intelligent manufacturing using social engineering optimizer, Symmetry, vol. 13, no. 4, p. 663, 2021.

[27]

M. L. Lee, S. Behdad, X. Liang, and M. Zheng, Task allocation and planning for product disassembly with human–robot collaboration, Robot. Comput. Integr. Manuf., vol. 76, p. 102306, 2022.

[28]

L. Guo, Z. Zhang, and X. Zhang, Human–robot collaborative partial destruction disassembly sequence planning method for end-of-life product driven by multi-failures, Adv. Eng. Inform., vol. 55, p. 101821, 2023.

[29]
P. Liang, Y. Fu, X. Guo, and L. Qi, Stochastic multi-product disassembly sequence planning, in Proc. IEEE 17th Int. Conf. Automation Science and Engineering (CASE), Lyon, France, 2021, pp. 1201–1206.
[30]

P. Liang, Y. Fu, K. Gao, and H. Sun, An enhanced group teaching optimization algorithm for multi-product disassembly line balancing problems, Complex Intell. Syst., vol. 8, no. 6, pp. 4497–4512, 2022.

[31]

X. Guo, Z. Zhang, L. Qi, S. Liu, Y. Tang, and Z. Zhao, Stochastic hybrid discrete grey wolf optimizer for multi-objective disassembly sequencing and line balancing planning in disassembling multiple products, IEEE Trans. Autom. Sci. Eng., vol. 19, no. 3, pp. 1744–1756, 2022.

[32]

X. Liu, F. Chu, F. Zheng, C. Chu, and M. Liu, Distributionally robust and risk-averse optimisation for the stochastic multi-product disassembly line balancing problem with workforce assignment, Int. J. Prod. Res., vol. 60, no. 6, pp. 1973–1991, 2022.

[33]

P. Hu, F. Chu, M. Liu, S. J. Wang, and P. Wu, An integrated approach for a new flexible multi-product disassembly line balancing problem, Comput. Oper. Res., vol. 148, p. 105932, 2022.

[34]

T. Yin, Z. Zhang, Y. Zhang, T. Wu, and W. Liang, Mixed-integer programming model and hybrid driving algorithm for multi-product partial disassembly line balancing problem with multi-robot workstations, Robot. Comput. Integr. Manuf., vol. 73, p. 102251, 2022.

[35]

J. S. Kim, J. H. Park, and D. H. Lee, Iterated greedy algorithms to minimize the total family flow time for job-shop scheduling with job families and sequence-dependent set-ups, Eng. Optim., vol. 49, no. 10, pp. 1719–1732, 2017.

[36]

K. Gao, L. Wang, J. Luo, H. Jiang, A. Sadollah, and Q. Pan, Discrete harmony search algorithm for scheduling and rescheduling the reprocessing problems in remanufacturing: A case study, Eng. Optim., vol. 50, no. 6, pp. 965–981, 2018.

[37]

J. Shi, W. Zhang, S. Zhang, and J. Chen, A new bifuzzy optimization method for remanufacturing scheduling using extended discrete particle swarm optimization algorithm, Comput. Ind. Eng., vol. 156, p. 107219, 2021.

[38]

X. Liu, J. Chen, X. Huang, S. Guo, S. Zhang, and M. Chen, A new job shop scheduling method for remanufacturing systems using extended artificial bee colony algorithm, IEEE Access, vol. 9, pp. 132429–132441, 2021.

[39]

W. Li, C. Zhang, C. Liu, and X. Liu, Error propagation model and optimal control method for the quality of remanufacturing assembly, J. Intell. Fuzzy Syst., vol. 42, no. 3, pp. 2533–2547, 2022.

[40]

C. Liu, Q. Zhu, F. Wei, W. Rao, J. Liu, J. Hu, and W. Cai, An integrated optimization control method for remanufacturing assembly system, J. Clean. Prod., vol. 248, p. 119261, 2020.

[41]

M. Hojati, Minimizing make-span in 2-stage disassembly flow-shop scheduling problem, Comput. Ind. Eng., vol. 94, pp. 1–5, 2016.

[42]

M. G. Kim, J. M. Yu, and D. H. Lee, Scheduling algorithms for remanufacturing systems with parallel flow-shop-type reprocessing lines, Int. J. Prod. Res., vol. 53, no. 6, pp. 1819–1831, 2015.

[43]

J. M. Kim, Y. D. Zhou, and D. H. Lee, Priority scheduling to minimize the total tardiness for remanufacturing systems with flow-shop-type reprocessing lines, Int. J. Adv. Manuf. Technol., vol. 91, no. 9, pp. 3697–3708, 2017.

[44]

W. Wang, G. Tian, H. Zhang, Z. Li, and L. Zhang, A hybrid genetic algorithm with multiple decoding methods for energy-aware remanufacturing system scheduling problem, Robot. Comput. Integr. Manuf., vol. 81, p. 102509, 2023.

[45]

J. M. Yu and D. H. Lee, Scheduling algorithms for job-shop-type remanufacturing systems with component matching requirement, Comput. Ind. Eng., vol. 120, pp. 266–278, 2018.

[46]

G. Gong, Q. Deng, R. Chiong, X. Gong, H. Huang, and W. Han, Remanufacturing-oriented process planning and scheduling: Mathematical modelling and evolutionary optimisation, Int. J. Prod. Res., vol. 58, no. 12, pp. 3781–3799, 2020.

[47]

X. Wen, K. Wang, H. Li, H. Sun, H. Wang, and L. Jin, A two-stage solution method based on NSGA-II for green multi-objective integrated process planning and scheduling in a battery packaging machinery workshop, Swarm Evol. Comput., vol. 61, p. 100820, 2021.

[48]

W. Zhang, Y. Zheng, and R. Ahmad, The integrated process planning and scheduling of flexible job-shop-type remanufacturing systems using improved artificial bee colony algorithm, J. Intell. Manuf., vol. 34, no. 7, pp. 2963–2988, 2023.

[49]

J. Shi, W. Zhang, S. Zhang, W. Wang, J. Lin, and R. Feng, A new environment-aware scheduling method for remanufacturing system with non-dedicated reprocessing lines using improved flower pollination algorithm, J. Manuf. Syst., vol. 57, pp. 94–108, 2020.

[50]

A. Ala, A. Goli, S. Mirjalili, and V. Simic, A fuzzy multi-objective optimization model for sustainable healthcare supply chain network design, Appl Soft Comput, vol. 150, p. 111012, 2024.

[51]

A. Ala, V. Simic, D. Pamucar, and N. Bacanin, Enhancing patient information performance in Internet of Things-based smart healthcare system: Hybrid artificial intelligence and optimization approaches, Eng. Appl. Artif. Intell., vol. 131, p. 107889, 2024.

[52]

J. Wang, D. Lei, and J. Cai, An adaptive artificial bee colony with reinforcement learning for distributed three-stage assembly scheduling with maintenance, Appl. Soft Comput., vol. 117, p. 108371, 2022.

[53]

R. Qi, J. Q. Li, J. Wang, H. Jin, and Y. Y. Han, QMOEA: A Q-learning-based multiobjective evolutionary algorithm for solving time-dependent green vehicle routing problems with time windows, Inf. Sci. Int. J., vol. 608, pp. 178–201, 2022.

[54]

J. J. Ji, Y. N. Guo, X. Z. Gao, D. W. Gong, and Y. P. Wang, Q-learning-based hyperheuristic evolutionary algorithm for dynamic task allocation of crowdsensing, IEEE Trans. Cybern., vol. 53, no. 4, pp. 2211–2224, 2023.

[55]

J. Cai, D. Lei, J. Wang, and L. Wang, A novel shuffled frog-leaping algorithm with reinforcement learning for distributed assembly hybrid flow shop scheduling, Int. J. Prod. Res., vol. 61, no. 4, pp. 1233–1251, 2023.

[56]

Y. Ren, K. Gao, Y. Fu, H. Sang, D. Li, and Z. Luo, A novel Q-learning based variable neighborhood iterative search algorithm for solving disassembly line scheduling problems, Swarm Evol. Comput., vol. 80, p. 101338, 2023.

[57]

M. Chu and W. Chen, Human-robot collaboration disassembly planning for end-of-life power batteries, J. Manuf. Syst., vol. 69, pp. 271–291, 2023.

[58]

G. Tian, M. Zhou, and P. Li, Disassembly sequence planning considering fuzzy component quality and varying operational cost, IEEE Trans. Autom. Sci. Eng., vol. 15, no. 2, pp. 748–760, 2018.

[59]

Y. Hou, Y. Fu, K. Gao, H. Zhang, and A. Sadollah, Modelling and optimization of integrated distributed flow shop scheduling and distribution problems with time windows, Expert Syst. Appl., vol. 187, p. 115827, 2022.

[60]

C. Bierwirth and D. C. Mattfeld, Production scheduling and rescheduling with genetic algorithms, Evol. Comput., vol. 7, no. 1, pp. 1–17, 1999.

[61]

Y. Fu, X. Ma, K. Gao, Z. Li, and H. Dong, Multi-objective home health care routing and scheduling with sharing service via a problem-specific knowledge-based artificial bee colony algorithm, IEEE Trans. Intell. Transp. Syst., vol. 25, no. 2, pp. 1706–1719, 2024.

[62]

Y. Yang, Y. Song, W. Guo, Q. Lei, A. Sun, and L. Fan, Guided shuffled frog-leaping algorithm for flexible job shop scheduling problem with variable sublots and overlapping in operations, Comput. Ind. Eng., vol. 180, p. 109209, 2023.

[63]

D. Z. Zheng and L. Wang, An effective hybrid heuristic for flow shop scheduling, Int. J. Adv. Manuf. Technol., vol. 21, no. 1, pp. 38–44, 2003.

[64]

L. Wang, Z. Pan, and J. Wang, A review of reinforcement learning based intelligent optimization for manufacturing scheduling, Complex System Modeling and Simulation, vol. 1, no. 4, pp. 257–270, 2021.

[65]

G. Tian, M. Zhou, and J. Chu, A chance constrained programming approach to determine the optimal disassembly sequence, IEEE Trans. Autom. Sci. Eng., vol. 10, no. 4, pp. 1004–1013, 2013.

[66]
D. C. Montgomery, Design and Analysis of Experiments. Hoboken, NJ, USA: John Wiley & Sons, 2017.
[67]

X. Ma, Y. Fu, K. Gao, L. Zhu, and A. Sadollah, A multi-objective scheduling and routing problem for home health care services via brain storm optimization, Complex System Modeling and Simulation, vol. 3, no. 1, pp. 32–46, 2023.

[68]

M. Friedman, The use of ranks to avoid the assumption of normality implicit in the analysis of variance, J. Am. Stat. Assoc., vol. 32, no. 200, pp. 675–701, 1937.

[69]

D. G. Pereira, A. Afonso, and F. M. Medeiros, Overview of Friedman’s test and post-hoc analysis, Commun. Stat. Simul. Comput., vol. 44, no. 10, pp. 2636–2653, 2015.

[70]

Z. Zhang, Y. Fu, K. Gao, H. Zhang, and L. Wang, A cooperative evolutionary algorithm with simulated annealing for integrated scheduling of distributed flexible job shops and distribution, Swarm Evol. Comput., vol. 85, p. 101467, 2024.

[71]
Z. Wu, R. Zhou, M. Goh, Y. Wang, Z. Xu, and W. Song, (DT4Smart) a digital twin-based modularized design approach for smart warehouses, Int. J. Comput. Integr. Manuf.
[72]

K. Z. Gao, Z. M. He, Y. Huang, P. Y. Duan, and P. N. Suganthan, A survey on meta-heuristics for solving disassembly line balancing, planning and scheduling problems in remanufacturing, Swarm Evol. Comput., vol. 57, p. 100719, 2020.

Complex System Modeling and Simulation
Pages 184-209
Cite this article:
Wang F, Fu Y, Gao K, et al. A Q-Learning Based Hybrid Meta-Heuristic for Integrated Scheduling of Disassembly and Reprocessing Processes Considering Product Structures and Stochasticity. Complex System Modeling and Simulation, 2024, 4(2): 184-209. https://doi.org/10.23919/CSMS.2024.0007

347

Views

35

Downloads

5

Crossref

6

Scopus

Altmetrics

Received: 25 February 2024
Revised: 10 April 2024
Accepted: 29 April 2024
Published: 30 June 2024
© The author(s) 2024.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return