Energy Efficiency Maximization in RISs-Assisted UAVs-Based Edge Computing Network Using Deep Reinforcement Learning
), Abstract
Edge Computing (EC) pushes computational capability to the Terrestrial Devices (TDs), providing more efficient and faster computing solutions. Unmanned Aerial Vehicles (UAVs) equipped with EC servers can be flexibly deployed, even in complex terrains, to provide mobile computing services at all times. Meanwhile, UAVs can establish an air-to-ground line-of-sight link with TDs to improve the quality of communication link. However, the UAV-to-TD link may be obstructed by ground obstacles such as buildings or trees, leading to sub-optimal data transmission rates. To surmount this issue, Reconfigurable Intelligent Surfaces (RISs) emerge as a promising technology capable of intelligently reflecting signals to enhance communication quality between UAVs and TDs. In this paper, we consider the RISs-assisted multi-UAVs collaborative edge Computing Network (RUCN) in urban environment, in which we study the computational offloading problem. Our goal is to maximize the overall energy efficiency of UAVs by jointly optimizing the flight duration and trajectories of UAVs, and the phase shifts of RISs. It is worth noting that this problem has been formally established as NP-hard. Therefore, we propose the Deep Deterministic Policy Gradients based UAV Trajectory and RIS Phase shift optimization algorithm (UTRP-DDPG) to solve this complex optimization challenge. The results of extensive numerical experiments show that the proposed algorithm outperforms the other benchmark algorithms under various parameter settings. Specially, the UTRP-DDPG algorithm improves the UAV energy efficiency by at least 2% compared to DQN algorithm.
Keywords
References
A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, Internet of Things: A survey on enabling technologies, protocols, and applications, IEEE Commun. Surv. Tutor., vol. 17, no. 4, pp. 2347–2376, 2015.
N. Cheng, J. He, Z. Yin, C. Zhou, H. Wu, F. Lyu, H. Zhou, and X. Shen, 6G service-oriented space-air-ground integrated network: A survey, Chin. J. Aeronaut., vol. 35, no. 9, pp. 1–18, 2022.
M. Asim, W. K. Mashwani, and A. A. Abd El-Latif, Energy and task completion time minimization algorithm for UAVs-empowered MEC system, Sustainable Computing: Informatics and Systems, vol. 35, p. 100698, 2022.
C. Luo, J. Zhang, X. Cheng, Y. Hong, Z. Chen, and X. Xing, Computation off-loading in resource-constrained edge computing systems based on deep reinforcement learning, IEEE Trans. Comput., vol. 73, no. 1, pp. 109–122, 2024.
J. Liu, J. Ren, Y. Zhang, X. Peng, Y. Zhang, and Y. Yang, Efficient dependent task offloading for multiple applications in MEC-cloud system, IEEE Trans. Mob. Comput., vol. 22, no. 4, pp. 2147–2162, 2023.
W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, Edge computing: Vision and challenges, IEEE Internet Things J., vol. 3, no. 5, pp. 637–646, 2016.
S. Shao, S. Liu, K. Li, S. You, H. Qiu, X. Yao, and Y. Ji, LBA-EC: Load balancing algorithm based on weighted bipartite graph for edge computing, Chin. J. Electron., vol. 32, no. 2, pp. 313–324, 2023.
C. Liu, W. Feng, Y. Chen, C. X. Wang, and N. Ge, Cell-free satellite-UAV networks for 6G wide-area Internet of Things, IEEE J. Sel. Areas Commun., vol. 39, no. 4, pp. 1116–1131, 2021.
X. Zhang, X. Wang, X. Xu, and Y. Zhao, Demand learning and cooperative deployment of UAV networks, Chin. J. Electron., vol. 31, no. 3, pp. 408–415, 2022.
H. Mei, K. Yang, Q. Liu, and K. Wang, 3D-trajectory and phase-shift design for RIS-assisted UAV systems using deep reinforcement learning, IEEE Trans. Veh. Technol., vol. 71, no. 3, pp. 3020–3029, 2022.
M. Mozaffari, W. Saad, M. Bennis, Y. H. Nam, and M. Debbah, A tutorial on UAVs for wireless networks: Applications, challenges, and open problems, IEEE Commun. Surv. Tutor., vol. 21, no. 3, pp. 2334–2360, 2019.
Q. Wu, L. Liu, and R. Zhang, Fundamental trade-offs in communication and trajectory design for UAV-enabled wireless network, IEEE Wirel. Commun., vol. 26, no. 1, pp. 36–44, 2019.
B. Di, H. Zhang, L. Song, Y. Li, Z. Han, and H. V. Poor, Hybrid beamforming for reconfigurable intelligent surface based multi-user communications: Achievable rates with limited discrete phase shifts, IEEE J. Sel. Areas Commun., vol. 38, no. 8, pp. 1809–1822, 2020.
L. Dong, Z. Liu, F. Jiang, and K. Wang, Joint optimization of deployment and trajectory in UAV and IRS-assisted IoT data collection system, IEEE Internet Things J., vol. 9, no. 21, pp. 21583–21593, 2022.
S. Li, B. Duo, M. Di Renzo, M. Tao, and X. Yuan, Robust secure UAV communications with the aid of reconfigurable intelligent surfaces, IEEE Trans. Wirel. Commun., vol. 20, no. 10, pp. 6402–6417, 2021.
S. Li, B. Duo, X. Yuan, Y. C. Liang, and M. Di Renzo, Reconfigurable intelligent surface assisted UAV communication: Joint trajectory design and passive beamforming, IEEE Wirel. Commun. Lett., vol. 9, no. 5, pp. 716–720, 2020.
P. A. Apostolopoulos, G. Fragkos, E. E. Tsiropoulou, and S. Papavassiliou, Data offloading in UAV-assisted multi-access edge computing systems under resource uncertainty, IEEE Trans. Mob. Comput., vol. 22, no. 1, pp. 175–190, 2023.
J. Xu, K. Ota, and M. Dong, Aerial edge computing: Flying attitude-aware collaboration for multi-UAV, IEEE Trans. Mob. Comput., vol. 22, no. 10, pp. 5706–5718, 2023.
Q. Luo, T. H. Luan, W. Shi, and P. Fan, Deep reinforcement learning based computation offloading and trajectory planning for multi-UAV cooperative target search, IEEE J. Sel. Areas Commun., vol. 41, no. 2, pp. 504–520, 2023.
Z. Ning, Y. Yang, X. Wang, L. Guo, X. Gao, S. Guo, and G. Wang, Dynamic computation offloading and server deployment for UAV-enabled multi-access edge computing, IEEE Trans. Mob. Comput., vol. 22, no. 5, pp. 2628–2644, 2023.
B. Duo, M. He, Q. Wu, and Z. Zhang, Joint dual-UAV trajectory and RIS design for ARIS-assisted aerial computing in IoT, IEEE Internet Things J., vol. 10, no. 22, pp. 19584–19594, 2023.
M. Asim, M. ELAffendi, and A. A. A. El-Latif, Multi-IRS and multi-UAV-assisted MEC system for 5G/6G networks: Efficient joint trajectory optimization and passive beamforming framework, IEEE Trans. Intell. Transp. Syst., vol. 24, no. 4, pp. 4553–4564, 2023.
Y. N. Shnaiwer, N. Kouzayha, M. Masood, M. Kaneko, and T. Y. Al-Naffouri, Multihop task routing in UAV-assisted mobile-edge computing IoT networks with intelligent reflective surfaces, IEEE Internet Things J., vol. 10, no. 8, pp. 7174–7188, 2023.
T. Ren, J. Niu, B. Dai, X. Liu, Z. Hu, M. Xu, and M. Guizani, Enabling efficient scheduling in large-scale UAV-assisted mobile-edge computing via hierarchical reinforcement learning, IEEE Internet Things J., vol. 9, no. 10, pp. 7095–7109, 2022.
X. Liu, Y. Liu, Y. Chen, and L. Hanzo, Trajectory design and power control for multi-UAV assisted wireless networks: A machine learning approach, IEEE Trans. Veh. Technol., vol. 68, no. 8, pp. 7957–7969, 2019.
C. Qiu, Y. Hu, Y. Chen, and B. Zeng, Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications, IEEE Internet Things J., vol. 6, no. 5, pp. 8577–8588, 2019.
京公网安备11010802044758号