AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (1.7 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Scheduling of Low-Latency Medical Services in Healthcare Cloud with Deep Reinforcement Learning

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
School of Information, Renmin University of China, Beijing 100872, China
Department of Computer Science and Technology, Nanjing University, Nanjing 210093, China
People’s Hospital of Quzhou City, Wenzhou Medical University, Quzhou 324000, China
Show Author Information

Abstract

In the current landscape of online data services, data transmission and cloud computing are often controlled separately by Internet Service Providers (ISPs) and cloud providers, resulting in significant cooperation challenges and suboptimal global data service optimization. In this study, we propose an end-to-end scheduling method aimed at supporting low-latency and computation-intensive medical services within local wireless networks and healthcare clouds. This approach serves as a practical paradigm for achieving low-latency data services in local private cloud environments. To meet the low-latency requirement while minimizing communication and computation resource usage, we leverage Deep Reinforcement Learning (DRL) algorithms to learn a policy for automatically regulating the transmission rate of medical services and the computation speed of cloud servers. Additionally, we utilize a two-stage tandem queue to address this problem effectively. Extensive experiments are conducted to validate the effectiveness for our proposed method under various arrival rates of medical services.

Keywords

medical servicetandem queuecloud computingDeep Reinforcement Learning (DRL)resource allocation

References

[1]
Marketsandmarkets, IoT in healthcare market by component, application, end user, and region global forecast to 2025, https://www.marketsandmarkets.com/Market-Reports/iot-healthcaremarket-160082804.html, 2020.
[2]
S. Dananjayan and G. M. Raj, 5G in healthcare: How fast will be the transformation? Ir. J. Med. Sci. 1971, vol. 190, no. 2, pp. 497–501, 2021.
Crossref
[3]

X. Wang and Z. Jin, An overview of mobile cloud computing for pervasive healthcare, IEEE Access, vol. 7, pp. 66774–66791, 2019.
Crossref Google Scholar
[4]
R. Zgheib, A. De Nicola, M. L. Villani, E. Conchon, and R. Bastide, A flexible architecture for cognitive sensing of activities in ambient assisted living, in Proc. IEEE 26th Int. Conf. Enabling Technologies : Infrastructure for Collaborative Enterprises (WETICE), Poznan, Poland, 2017, pp. 284–289.
Crossref
[5]

M. Tariq, H. Majeed, M. O. Beg, F. A. Khan, and A. Derhab, Accurate detection of sitting posture activities in a secure IoT based assisted living environment, Future Gener. Comput. Syst., vol. 92, pp. 745–757, 2019.
Crossref Google Scholar
[6]
A. Rashed, A. Ibrahim, A. Adel, B. Mourad, A. Hatem, M. Magdy, N. Elgaml, and A. Khattab, Integrated IoT medical platform for remote healthcare and assisted living, in Proc. Japan-Africa Conf. Electronics, Communications and Computers (JAC-ECC), Alexandria, Egypt, 2017, pp. 160–163.
Crossref
[7]

G. Marques and R. Pitarma, An indoor monitoring system for ambient assisted living based on Internet of Things architecture, Int. J. Environ. Res. Public Health, vol. 13, no. 11, p. 1152, 2016.
Crossref Google Scholar
[8]

C. Ma, X. Dai, J. Zhu, N. Liu, H. Sun, and M. Liu, DrivingSense: dangerous driving behavior identification based on smartphone autocalibration, Mob. Inf. Syst., vol. 2017, p. 9075653, 2017.
Crossref Google Scholar
[9]

Y. Liu, L. Zhang, Y. Yang, L. Zhou, L. Ren, F. Wang, R. Liu, Z. Pang, and M. J. Deen, A novel cloud-based framework for the elderly healthcare services using digital twin, IEEE Access, vol. 7, pp. 49088–49101, 2019.
Crossref Google Scholar
[10]

H. Mora, F. J. Mora Gimeno, M. T. Signes-Pont, and B. Volckaert, Multilayer architecture model for mobile cloud computing paradigm, Complexity, vol. 2019, p. 3951495, 2019.
Crossref Google Scholar
[11]

S. Shukla, M. F. Hassan, D. C. Tran, R. Akbar, I. V. Paputungan, and M. K. Khan, Improving latency in Internet-of-Things and cloud computing for real-time data transmission: A systematic literature review (SLR), Clust. Comput., vol. 26, no. 5, pp. 2657–2680, 2023.
Crossref Google Scholar
[12]
R. Mahmud, F. L. Koch, and R. Buyya, Cloud-fog interoperability in IoT-enabled healthcare solutions, in Proc. 19th Int. Conf. Distributed Computing and Networking, Varanasi, India, 2018, pp. 1–10.
Crossref
[13]

C. S. Nandyala and H.-K. Kim, From cloud to fog and IoT-based real-time U-healthcare monitoring for smart homes and hospitals, Int. J. Smart Home, vol. 10, no. 2, pp. 187–196, 2016.
Crossref Google Scholar
[14]

K. S. Awaisi, S. Hussain, M. Ahmed, A. Ali Khan, and G. Ahmed, Leveraging IoT and fog computing in healthcare systems, IEEE Internet Things Mag., vol. 3, no. 2, pp. 52–56, 2020.
Crossref Google Scholar
[15]

S. Tuli, N. Basumatary, S. S. Gill, M. Kahani, R. C. Arya, G. S. Wander, and R. Buyya, HealthFog: An ensemble deep learning based smart healthcare system for automatic diagnosis of heart diseases in integrated IoT and fog computing environments, Future Gener. Comput. Syst., vol. 104, pp. 187–200, 2020.
Crossref Google Scholar
[16]

J. Ren, J. Li, H. Liu, and T. Qin, Task offloading strategy with emergency handling and blockchain security in SDN-empowered and fog-assisted healthcare IoT, Tsinghua Science and Technology, vol. 27, no. 4, pp. 760–776, 2022.
Crossref Google Scholar
[17]

Y. Wang, X. Tao, Y. T. Hou, and P. Zhang, Effective capacity-based resource allocation in mobile edge computing with two-stage tandem queues, IEEE Trans. Commun., vol. 67, no. 9, pp. 6221–6233, 2019.
Crossref Google Scholar
[18]

C. Ma, J. Zhu, M. Liu, H. Zhao, N. Liu, and X. Zou, Parking edge computing: Parked-vehicle-assisted task offloading for urban VANETs, IEEE Internet Things J., vol. 8, no. 11, pp. 9344–9358, 2021.
Crossref Google Scholar
[19]
M. Yang, N. Liu, L. Zuo, Y. Feng, M. Liu, H. Gong, and M. Liu, Dynamic charging scheme problem with actor-critic reinforcement learning, IEEE Internet Things J., vol. 8, no. 1, pp. 370–380, 2021.
Crossref
[20]
S. P. Erdeniz, I. Maglogiannis, A. Menychtas, A. Felfernig, and T. N. T. Tran, Recommender systems for iot enabled m-health applications, in Proc. IFIP International Conference on Artificial Intelligence Applications and Innovations, Cham, Switzerland: Springer, 2018: 227–237.
Crossref
[21]

B. Xu, L. Xu, H. Cai, L. Jiang, Y. Luo, and Y. Gu, The design of an m-health monitoring system based on a cloud computing platform, Enterp. Inf. Syst., vol. 11, no. 1, pp. 17–36, 2017.
Crossref Google Scholar
[22]
A. Kelati, I. B. Dhaou, and H. Tenhunen, Biosignal monitoring platform using wearable IoT, in Proceedings of the 22st Conference of Open Innovations Association FRUCT, Petrozavodsk, Russia, 2018, pp. 9–13.
[23]
H. Xu, L. Zuo, F. Sun, M. Yang, and N. Liu, Low-latency patient monitoring service for cloud computing based healthcare system by applying reinforcement learning, in Proc. IEEE 8th Int. Conf. Computer and Communications (ICCC), Chengdu, China, 2022, pp. 1373–1377.
Crossref
[24]

M. Raeis, A. Tizghadam, and A. Leon-Garcia, Queue-learning: A reinforcement learning approach for providing quality of service, Proc. AAAI Conf. Artif. Intell., vol. 35, no. 1, pp. 461–468, 2021.
Crossref Google Scholar
[25]
A. M. Law, Simulation Modeling & Analysis, 5th ed. New York, NY, USA: McGraw-Hill, 2015.
[26]
M. Raeis, A. Tizghadam, and A. Leon-Garcia, Reinforcement learning-based admission control in delay-sensitive service systems, in Proc. GLOBECOM 2020 - 2020 IEEE Global Communications Conf., Taipei, China, 2020, pp. 1–6.
Crossref
[27]
P. R. Montague Reinforcement learning: An introduction, by Sutton, RS, and Barto, AG, Trends Cogn. Sci., vol. 3, no. 9, p. 360, 1999.
Crossref
[28]
C. J. Watkins and P. Dayan, Q-learning, Machine learning, vol. 8, nos. 3&4, pp. 279–292, 1992.
Crossref
[29]

V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., Human-level control through deep reinforcement learning, Nature, vol. 518, no. 7540, pp. 529–533, 2015.
Crossref Google Scholar
[30]
R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour, Policy gradient methods for reinforcement learning with function approximation, in Proc. of 12th Int Conf. on Neural Information Processing Systems, Denver, CO, USA, 2000, pp. 1057–1063.
[31]

N. Metropolis and S. Ulam, The Monte Carlo method, J. Am. Stat. Assoc., vol. 44, no. 247, pp. 335–341, 1949.
Crossref Google Scholar
[32]
T. Xie, X. Cheng, X. Wang, M. Liu, J. Deng, T. Zhou, and M. Liu, Cut-thumbnail: A novel data augmentation for convolutional neural network, in Proc. 29th ACM Int. Conf. Multimedia. Virtual Event China, 2021, pp. 1627–1635.
Crossref
[33]

L. Wu, M. Liu, X.-M. Wang, G.-H. Chen, and H.-G. Gong, Mobile distribution-aware data dissemination for vehicular ad hoc networks, J. Softw., vol. 22, no. 7, pp. 1580–1596, 2011.
Crossref Google Scholar
Tsinghua Science and Technology
Volume 30 Issue 1,
February 2025
Pages 100-111
DOI: 10.26599/TST.2024.9010033
Cite this article:
Du H, Liu M, Liu N, et al. Scheduling of Low-Latency Medical Services in Healthcare Cloud with Deep Reinforcement Learning. Tsinghua Science and Technology, 2025, 30(1): 100-111. https://doi.org/10.26599/TST.2024.9010033

267

Views

47

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 26 October 2023
Revised: 16 January 2024
Accepted: 03 February 2024
Published: 11 September 2024
© The Author(s) 2025.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return