AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Intelligent and Converged Networks Article
PDF (7.1 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

A deep Q-learning model for sequential task offloading in edge AI systems

Department of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610000, China
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 515100, China
Department of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610000, China
Show Author Information

Abstract

Currently, edge Artificial Intelligence (AI) systems have significantly facilitated the functionalities of intelligent devices such as smartphones and smart cars, and supported diverse applications and services. This fundamental supports come from continuous data analysis and computation over these devices. Considering the resource constraints of terminal devices, multi-layer edge artificial intelligence systems improve the overall computing power of the system by scheduling computing tasks to edge and cloud servers for execution. Previous efforts tend to ignore the nature of strong pipelined characteristics of processing tasks in edge AI systems, such as the encryption, decryption and consensus algorithm supporting the implementation of Blockchain techniques. Therefore, this paper proposes a new pipelined task scheduling algorithm (referred to as PTS-RDQN), which utilizes the system representation ability of deep reinforcement learning and integrates multiple dimensional information to achieve global task scheduling. Specifically, a co-optimization strategy based on Rainbow Deep Q-Learning (RainbowDQN) is proposed to allocate computation tasks for mobile devices, edge and cloud servers, which is able to comprehensively consider the balance of task turnaround time, link quality, and other factors, thus effectively improving system performance and user experience. In addition, a task scheduling strategy based on PTS-RDQN is proposed, which is capable of realizing dynamic task allocation according to device load. The results based on many simulation experiments show that the proposed method can effectively improve the resource utilization, and provide an effective task scheduling strategy for the edge computing system with cloud-edge-end architecture.

Keywords

edge computingtask schedulingreinforcement learningRainbow Deep Q-Learning (RainbowDQN)

References

[1]

R. C. Xie, X. F. Lian, Q. M. Jia, T. Huang, and Y. Liu, Overview of mobile edge computing offload technology, J. Commun, vol. 39, no. 11, pp. 138–155, 2018.
Google Scholar
[2]

T. X. Tran and D. Pompili, Joint task offloading and resource allocation for multi-server mobile-edge computing networks, IEEE Trans. Veh. Technol, vol. 68, no. 1, pp. 856–868, 2018.
Crossref Google Scholar
[3]

H. B. Zhang, H. Li, S. X. Chen, and X. F. He, Task scheduling and resource optimization based on mobile edge computing in ultra-dense networks, J. Electron. Inf., vol. 41, pp. 1194–1201, 2019.
Google Scholar
[4]

W. Liu, Y. Huang, W. Du, and W. Wang, Resource-constrained serial task offload strategy in mobile edge computing, J. Softw., vol. 31, no. 6, pp. 1889–1908, 2020.
Google Scholar
[5]

Z. Cai and X. Zheng, A private and efficient mechanism for data uploading in smart cyber-physical systems, IEEE Trans. Netw. Sci. Eng., vol. 7, no. 2, pp. 766–775, 2020.
Crossref Google Scholar
[6]

H. B. Zhao, T. W. Zhang, Y. Chen, H. L. Zhao, and H. B. Zhu, DQN-based task distribution offloading algorithm for vehicular edge networks, Journal of Communications, vol. 41, pp. 172–178, 2020.
Google Scholar
[7]

H. F. Lu, C. H. Gu, F. Luo, W. C. Ding, T. Yang, and S. Zheng, Research on mobile edge computing task scheduling based on deep reinforcement learning, Computer Research and Development, vol. 57, pp. 1539–1554, 2020.
Google Scholar
[8]
L. Huang, X. Feng, A. Feng, Y. Huang, and L. P. Qian, Distributed deep learning-based offloading for mobile edge computing networks, Mobile networks and applications, pp. 1–8. 2018.
Crossref
[9]

H. B. Zhao, Q. Y. Luan, J. Zhu, and X. F. He, V2X task unloading scheme based on mobile edge computing, Journal of Electronics and Information Technology, vol. 40, no. 11, pp. 2736–2743, 2018.
Google Scholar
[10]

Y. J. Wang, J. F. Fan, and C. Y. Wang, Two-stage task offloading strategy based on game theory in cloud-edge environment, J. Computer Applications, vol. 41, no. 5, pp. 1392, 2021.
Google Scholar
[11]

X. Zheng and Z. Cai, Privacy-preserved data sharing towards multiple parties in industrial IoTs, IEEE Journal on Selected Areas in Comuunications, vol. 38, no. 5, p. pp. 968–979, 2020.
Crossref Google Scholar
[12]

X. Chen, Decentralized computation offloading game for mobile cloud computing, IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 4, pp. 974–983, 2015.
Crossref Google Scholar
[13]

Z. Cai, X. Zheng, J. Wang, and Z. He, Private data trading towards range counting queries in Internet of Things, IEEE Trans. Mob. Comput., vol. 22, no. 8, pp. 4881–4897, 2023.
Crossref Google Scholar
[14]

A. Shakarami, M. Ghobaei-Arani, and A. Shahidinejad, A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective, Comput. Netw, vol. 182, p. 107496, 2020.
Crossref Google Scholar
[15]
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller, H. Y. Ong, K. Chavez, and A. Hong, Playing atari with deep reinforcement learning, arXiv preprint arXiv: 1312.5602, 2013.
[16]
T. Schaul, J. Quan, I. Antonoglou, and D. Silver, Prioritized experience replay, arXiv preprint arXiv: 1511.05952, 2015.
[17]

X. Zheng, L. Tian, and Z. Cai, A fair and rational data sharing strategy toward two-stage industrial Internet of Things, IEEE Trans. Ind. Inform., vol. 19, no. 1, pp. 1088–1096, 2023.
Crossref Google Scholar
[18]
H. Van Hasselt, A. Guez, and D. Silver, Deep reinforcement learning with double q-learning, in proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, pp. 2094–2100, 2016.
Crossref
[19]
Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, Dueling network architectures for deep reinforcement learning, in international conference on machine learning, vol. 48, no. 1, pp. 1995–2003, 2016.
[20]
M. G. Bellemare, W. Dabney, and R. Munos, A distributional perspective on reinforcement learning, arXiv preprint arXiv: 1707.06887, 2017.
[21]
M. Fortunato, M. G. Azar, B. Piot, J. Menick, I. Osband, A. Graves, V. Mnih, R. Munos, D. Hassabis, O. Pietquin, et al., Noisy networks for exploration, arXiv preprint arXiv: 1706.10295, 2017.
Intelligent and Converged Networks
Volume 5 Issue 3,
September 2024
Pages 207-221
DOI: 10.23919/ICN.2024.0015
Cite this article:
Liu D, Gu S, Fan X, et al. A deep Q-learning model for sequential task offloading in edge AI systems. Intelligent and Converged Networks, 2024, 5(3): 207-221. https://doi.org/10.23919/ICN.2024.0015

78

Views

6

Downloads

0

Crossref

0

Scopus

Altmetrics
Received: 04 February 2024
Revised: 26 March 2024
Accepted: 16 April 2024
Published: 30 September 2024
© All articles included in the journal are copyrighted to the ITU and TUP.

This work is available under the CC BY-NC-ND 3.0 IGO license:https://creativecommons.org/licenses/by-nc-nd/3.0/igo/
Return