AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (4.6 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Optimizing Risk-Aware Task Migration Algorithm Among Multiplex UAV Groups Through Hybrid Attention Multi-Agent Reinforcement Learning

School of Computer Science and Engineering, Southeast University, Nanjing 211189
School of Software Engineering, Southeast University, Nanjing 211189
School of Cyber Science and Engineering, Southeast University, Nanjing 211189
PredictHQ, Auckland 1010, New Zealand

Show Author Information

Abstract

Recently, with the increasing complexity of multiplex Unmanned Aerial Vehicles (multi-UAVs) collaboration in dynamic task environments, multi-UAVs systems have shown new characteristics of inter-coupling among multiplex groups and intra-correlation within groups. However, previous studies often overlooked the structural impact of dynamic risks on agents among multiplex UAV groups, which is a critical issue for modern multi-UAVs communication to address. To address this problem, we integrate the influence of dynamic risks on agents among multiplex UAV group structures into a multi-UAVs task migration problem and formulate it as a partially observable Markov game. We then propose a Hybrid Attention Multi-agent Reinforcement Learning (HAMRL) algorithm, which uses attention structures to learn the dynamic characteristics of the task environment, and it integrates hybrid attention mechanisms to establish efficient intra- and inter-group communication aggregation for information extraction and group collaboration. Experimental results show that in this comprehensive and challenging model, our algorithm significantly outperforms state-of-the-art algorithms in terms of convergence speed and algorithm performance due to the rational design of communication mechanisms.

Keywords

Unmanned Aerial Vehicle (UAV)multiplex UAV group structurestask migrationmulti-agent reinforcement learning

References

[1]
Y. Jiang, K. Di, Z. Hu, F. Chen, P. Li, and Y. Jiang, ε-maximum critic deep deterministic policy gradient for multi-agent reinforcement learning, in Proc. Int. Conf. on Parallel and Distributed Computing : Applications and Technologies, Singapore, 2024, pp. 180–189.
Crossref
[2]

Y. Pan, Q. Ran, Y. Zeng, B. Ma, J. Tang, and L. Cao, Symmetric Bayesian personalized ranking with softmax weight, IEEE Trans. Syst. Man Cybern, Syst., vol. 53, no. 7, pp. 4314–4323, 2023.
Crossref Google Scholar
[3]
F. Yan and K. Di, Multi-robot task allocation in the environment with functional tasks, in Proc. Thirty-First Int. Joint Conf. Artificial Intelligence, Vienna, Austria, 2022.
Crossref
[4]

F. Yan and K. Di, Solving the Multi-robot task allocation with functional tasks based on a hyper-heuristic algorithm, Appl. Soft Comput., vol. 146, p. 110628, 2023.
Crossref Google Scholar
[5]

L. Panait and S. Luke, Cooperative multi-agent learning: The state of the art, Auton. Agents Multi-Agent Syst., vol. 11, no. 3, pp. 387–434, 2005.
Crossref Google Scholar
[6]
E. Shieh, B. An, R. Yang, M. Tambe, C. Baldwin, J. DiRenzo, B. Maule, and G. Meyer, Protect: A deployed game theoretic system to protect the ports of the United States, presented at the 11th Conference in autonomous agents and multiagent systems, Minneapolis, MN, USA, 2012.
[7]

I. Tkach and S. Amador, Towards addressing dynamic multi-agent task allocation in law enforcement, Auton. Agents Multi-Agent Syst., vol. 35, no. 1, p. 11, 2021.
Crossref Google Scholar
[8]

D. Chen, M. R. Hajidavalloo, Z. Li, K. Chen, Y. Wang, L. Jiang, and Y. Wang, Deep multi-agent reinforcement learning for highway on-ramp merging in mixed traffic, IEEE Trans. Intell. Transport. Syst., vol. 24, no. 11, pp. 11623–11638, 2023.
Crossref Google Scholar
[9]

J. Ko, J. Jang, and C. Oh, A multi-agent driving simulation approach for evaluating the safety benefits of connected vehicles, IEEE Trans. Intell. Transport. Syst., vol. 23, no. 5, pp. 4512–4524, 2022.
Crossref Google Scholar
[10]

J. Zhang, Y. Cui, and J. Ren, Dynamic mission planning algorithm for UAV formation in battlefield environment, IEEE Trans. Aerosp. Electron. Syst., vol. 59, no. 4, pp. 3750–3765, 2023.
Crossref Google Scholar
[11]
J. Guo, H. Gao, Z. Liu, F. Huang, J. Zhang, X. Li, and J. Ma, ICRA: An intelligent clustering routing approach for UAV ad hoc networks, IEEE Trans. Intell. Transport. Syst., vol. 24, no. 2, pp. 2447–2460, 2023.
Crossref
[12]

B. Liu, W. Zhang, W. Chen, H. Huang, and S. Guo, Online computation offloading and traffic routing for UAV swarms in edge-cloud computing, IEEE Trans. Veh. Technol., vol. 69, no. 8, pp. 8777–8791, 2020.
Crossref Google Scholar
[13]

A. Kaur and K. Kumar, Energy-efficient resource allocation in cognitive radio networks under cooperative multi-agent model-free reinforcement learning schemes, IEEE Trans. Netw. Serv. Manage., vol. 17, no. 3, pp. 1337–1348, 2020.
Crossref Google Scholar
[14]
M. Hua, Y. Wang, C. Li, Y. Huang, and L. Yang, UAV-aided mobile edge computing systems with one by one access scheme, IEEE Trans. Green Commun. Netw., vol. 3, no. 3, pp. 664–678, 2019.
Crossref
[15]

J. Wang, K. Liu, and J. Pan, Online UAV-mounted edge server dispatching for mobile-to-mobile edge computing, IEEE Internet Things J., vol. 7, no. 2, pp. 1375–1386, 2020.
Crossref Google Scholar
[16]

A. Sacco, F. Esposito, and G. Marchetto, Resource inference for sustainable and responsive task offloading in challenged edge networks, IEEE Trans. Green Commun. Netw., vol. 5, no. 3, pp. 1114–1127, 2021.
Crossref Google Scholar
[17]

B. Liu, W. Zhang, W. Chen, H. Huang, and S. Guo, Online computation offloading and traffic routing for UAV swarms in edge-cloud computing, IEEE Trans. Veh. Technol., vol. PP, no. 99, p. 1, 2020.
Crossref Google Scholar
[18]

Y. Liu, H. Yu, S. Xie, and Y. Zhang, Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks, IEEE Trans. Veh. Technol., vol. 68, no. 11, pp. 11158–11168, 2019.
Crossref Google Scholar
[19]

Z. Gao, L. Yang, and Y. Dai, Large-scale computation offloading using a multi-agent reinforcement learning in heterogeneous multi-access edge computing, IEEE Trans. Mobile Comput., vol. 22, no. 6, pp. 3425–3443, 2023.
Crossref Google Scholar
[20]

X. Zhu, Y. Luo, A. Liu, M. Z. A. Bhuiyan, and S. Zhang, Multiagent deep reinforcement learning for vehicular computation offloading in IoT, IEEE Internet Things J., vol. 8, no. 12, pp. 9763–9773, 2021.
Crossref Google Scholar
[21]
R. Lowe, Y. I. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, Multiagent actor-critic for mixed cooperative-competitive environments, presented at the 31th Conference on Neural Information Processing Systems, California, CA, USA, 2017.
[22]
A. Melin, Ocean Wave Prediction with LSTM, https://www.kaggle.com/code/akdagmelih/ocean-waveprediction-with-lstm, 2020.
[23]

Y. Jiang, Y. Zhou, and Y. Li, Reliable task allocation with load balancing in multiplex networks, ACM Trans. Auton. Adapt. Syst., vol. 10, no. 1, p. 3, 2015.
Crossref Google Scholar
[24]

W. Zhang, Y. Wen, and D. O. Wu, Collaborative task execution in mobile cloud computing under a stochastic wireless channel, IEEE Trans. Wirel. Commun., vol. 14, no. 1, pp. 81–93, 2015.
Crossref Google Scholar
[25]
S. Iqbal and F. Sha, Actor-attention-critic for multi-agent reinforcement learning, arXiv preprint arXiv: 1810.02912, 2018.
[26]
H. Ryu, H. Shin, and J. Park, Multiagent actor-critic with hierarchical graph attention network, presented at the 34th Conference on Association for the Advancement of Artificial Intelligence, New York, NY, USA, 2020.
Crossref
Tsinghua Science and Technology
Volume 30 Issue 1,
February 2025
Pages 318-330
DOI: 10.26599/TST.2024.9010013
Cite this article:
Jiang Y, Di K, Qian R, et al. Optimizing Risk-Aware Task Migration Algorithm Among Multiplex UAV Groups Through Hybrid Attention Multi-Agent Reinforcement Learning. Tsinghua Science and Technology, 2025, 30(1): 318-330. https://doi.org/10.26599/TST.2024.9010013

121

Views

11

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 06 November 2023
Revised: 25 December 2023
Accepted: 04 January 2024
Published: 01 April 2024
© The Author(s) 2025.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return