AI Chat Paper
Discover the SciOpen Platform and Achieve Your Research Goals with Ease.
· A novel deep reinforcement learning approach to safe and efficient AAM traffic separation.
· A new MARL framework for AAM vehicle coordination in merging and intersection scenarios.
· Trade-off studies that reveal impacts of network design and hyperparameters on the performance of the algorithms.
· Extensive simulations that demonstrate the performance of the proposed methods.
Advanced Air Mobility (AAM) has emerged as a pioneering concept designed to optimize the efficacy and ecological sustainability of air transportation. Its core objective is to provide highly automated air transportation services for passengers or cargo, operating at low altitudes within urban, suburban, and rural regions. AAM seeks to enhance the efficiency and environmental viability of the aviation sector by revolutionizing the way air travel is conducted. In a complex aviation environment, traffic management and control are essential technologies for safe and effective AAM operations. One of the most difficult obstacles in the envisioned AAM systems is vehicle coordination at merging points and intersections. The escalating demand for air mobility services, particularly within urban areas, poses significant complexities to the execution of such missions. In this study, we propose a novel multi-agent reinforcement learning (MARL) approach to efficiently manage high-density AAM operations in structured airspace. Our approach provides effective guidance to AAM vehicles, ensuring conflict avoidance, mitigating traffic congestion, reducing travel time, and maintaining safe separation. Specifically, intelligent learning-based algorithms are developed to provide speed guidance for each AAM vehicle, ensuring secure merging into air corridors and safe passage through intersections. To validate the effectiveness of our proposed model, we conduct training and evaluation using BlueSky, an open-source air traffic control simulation environment. Through the simulation of thousands of aircraft and the integration of real-world data, our study demonstrates the promising potential of MARL in enabling safe and efficient AAM operations. The simulation results validate the efficacy of our approach and its ability to achieve the desired outcomes.
Hasan S. Urban air mobility (uam) market study. Tech Rep 2019.
Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9(8):1735-80.
Erzberger H, Heere K. Algorithm and operational concept for resolving short-range conflicts. Proc Inst Mech Eng G J Aerosp Eng 2010;224(2):225-43.
Chu T, Wang J, Codecà L, Li Z. Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Trans Intell Transport Syst 2019;21(3):1086-95.
Schuchardt BI, Geister D, Lüken T, Knabe F, Metz IC, Peinecke N, et al. Air traffic management as a vital part of urban air mobility—a review of dlr’s research work from 1995 to 2022. Aerospace 2023;10(1):81.
Pinto Neto EC, Baum DM, Almeida Jr JRd, Camargo Jr JB, Cugnasca PS. Deep learning in air traffic management (atm): a survey on applications, opportunities, and open challenges. Aerospace 2023;10(4):358.
Nachum O, Norouzi M, Xu K, Schuurmans D. Bridging the gap between value and policy based reinforcement learning. Adv Neural Inf Process Syst 2017;30.
Peters J. Policy gradient methods. Scholarpedia 2010;5(11):3698.
Sutton RS, McAllester D, Singh S, Mansour Y. Policy gradient methods for reinforcement learning with function approximation. Adv Neural Inf Process Syst 1999;12.
Morimura T, Uchibe E, Doya K. Natural actor-critic with baseline adjustment for variance reduction. Artif Life Robot 2008;13(1):275-9.
Busoniu L, Babuska R, De Schutter B. A comprehensive survey of multiagent reinforcement learning. IEEE Trans Sys Man Cybernet Part C (Applications and Reviews) 2008;38(2):156-72.
Matignon L, Laurent GJ, Le Fort-Piat N. Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems. Knowl Eng Rev 2012;27(1):1-31.
Kraemer L, Banerjee B. Multi-agent reinforcement learning as a rehearsal for decentralized planning. Neurocomputing 2016;190:82-94.
Weiss G. Multiagent systems: a modern approach to distributed artificial intelligence. Int J Comput Intell Appl 2001;1:331-4.
Niu Z, Zhong G, Yu H. A review on the attention mechanism of deep learning. Neurocomputing 2021;452:48-62.
Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res 2012;13(2).
Probst P, Wright MN, Boulesteix A-L. Hyperparameters and tuning strategies for random forest. Wiley Interdisciplinary Rev: Data Min Knowl Discov 2019;9(3):e1301.
Wu J, Chen X-Y, Zhang H, Xiong L-D, Lei H, Deng S-H. Hyperparameter optimization for machine learning models based on bayesian optimization. J Electr Sci Tech 2019;17(1):26-40.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
