AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Intelligent and Connected Vehicles Article
PDF (3.5 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Kinematics-aware multigraph attention network with residual learning for heterogeneous trajectory prediction

Zihao Sheng, Zilin Huang, Sikai Chen( )
Department of Civil and Environmental Engineering, University of Wisconsin–Madison, Madison, WI 53706, USA
Show Author Information

Abstract

Trajectory prediction for heterogeneous traffic agents plays a crucial role in ensuring the safety and efficiency of automated driving in highly interactive traffic environments. Numerous studies in this area have focused on physics-based approaches because they can clearly interpret the dynamic evolution of trajectories. However, physics-based methods often suffer from limited accuracy. Recent learning-based methods have demonstrated better performance, but they cannot be fully trusted due to the insufficient incorporation of physical constraints. To mitigate the limitations of purely physics-based and learning-based approaches, this study proposes a kinematics-aware multigraph attention network (KA-MGAT) that incorporates physics models into a deep learning framework to improve the learning process of neural networks. Besides, we propose a residual prediction module to further refine the trajectory predictions and address the limitations arising from simplified assumptions in kinematic models. We evaluate our proposed model through experiments on two challenging trajectory datasets, namely, ApolloScape and NGSIM. Our findings from the experiments demonstrate that our model outperforms various kinematics-agnostic models with respect to prediction accuracy and learning efficiency.

Keywords

trajectory predictionphysics-informed deep learningmultigraph attentionresidual learningautomated driving.

References

 
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Li, F. F., Savarese, S., 2016. Social LSTM: Human trajectory prediction in crowded spaces. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 961–971.
Crossref
 

Bai, L., An, Y., Sun, Y., 2023a. Measurement of project portfolio benefits with a GA-BP neural network group. IEEE Trans Eng Manage, 71, 4737−4749.
Crossref Google Scholar
 

Bai, L., Song, C., Zhou, X., Tian, Y., Wei, L., 2023b. Assessing project portfolio risk via an enhanced GA-BPNN combined with PCA. Eng Appl Artif Intell, 126, 106779.
Crossref Google Scholar
 
Batkovic, I., Zanon, M., Lubbe, N., Falcone, P., 2018. A computationally efficient model for pedestrian motion prediction. In: 2018 European Control Conference (ECC), 374–379.
Crossref
 

Cao, L., Luo, Y., Wang, Y., Chen, J., He, Y., 2023. Vehicle sideslip trajectory prediction based on time-series analysis and multiphysical model fusion. J Intell Connect Veh, 6, 161−172.
Crossref Google Scholar
 
Carrasco, S., Llorca, D. F., Sotelo, M. A., 2021. SCOUT: socially COnsistent and UndersTandable graph attention network for trajectory prediction of vehicles and VRUs. In: 2021 IEEE Intelligent Vehicles Symposium (IV), 1501–1508.
Crossref
 

Chen, Q., Wei, Z., Wang, X., Li, L., Lv, Y., 2022. Social relation and physical lane aggregator: Integrating social and physical features for multimodal motion prediction. J Intell Connect Veh, 5, 302−308.
Crossref Google Scholar
 

Chen, S., Dong, J., Ha, P. Y. J., Li, Y., Labi, S., 2021. Graph neural network and reinforcement learning for multiagent cooperative control of connected autonomous vehicles. Comput. Aided Civ. Infrastruct Eng, 36, 838−857.
Crossref Google Scholar
 

Chen, S., Zong, S., Chen, T., Huang, Z., Chen, Y., Labi, S., 2023. A taxonomy for autonomous vehicles considering ambient road infrastructure. Sustainability, 15, 11258.
Crossref Google Scholar
 
Cui, H., Nguyen, T., Chou, F. C., Lin, T. H., Schneider, J., Bradley, D., et al., 2020. Deep kinematic models for kinematically feasible vehicle trajectory predictions. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), 10563–10569.
Crossref
 
Deo, N., Trivedi, M. M., 2018. Convolutional social pooling for vehicle trajectory prediction. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 1468–1476.
Crossref
 

Di, X., Shi, R., Mo, Z., Fu, Y., 2023. Physics-informed deep learning for traffic state estimation: A survey and the outlook. Algorithms, 16, 305−340.
Crossref Google Scholar
 

Ding, Z., Zhao, H., 2023. Incorporating driving knowledge in deep learning based vehicle trajectory prediction: A survey. IEEE Trans Intell Veh, 8, 3996−4015.
Crossref Google Scholar
 

Dong, J., Chen, S., Miralinaghi, M., Chen, T., Labi, S., 2022. Development and testing of an image transformer for explainable autonomous driving systems. J Intell Connect Veh, 5, 235−249.
Crossref Google Scholar
 

Dong, J., Chen, S., Miralinaghi, M., Chen, T., Li, P., Labi, S., 2023. Why did the AI make that decision? Towards an explainable artificial intelligence (XAI) for autonomous driving systems. Transp Res Part C Emerg Technol, 156, 104358.
Crossref Google Scholar
 
Du, R., Chen, S., Dong, J., Chen, T., Fu, X., Labi, S., 2023. Dynamic urban traffic rerouting with fog-cloud reinforcement learning. Comput Aided Civ Infrastruct Eng, 1–21.
Crossref
 
Fang, L., Jiang, Q., Shi, J., Zhou, B., 2020. TPNet: trajectory proposal network for motion prediction. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6797–6806.
Crossref
 
Fernando, T., Denman, S., Sridharan, S., Fookes, C., 2019. Neighourhood context embeddings in deep inverse reinforcement learning for predicting pedestrian motion over long time horizons. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 1–9.
Crossref
 

González, D., Pérez, J., Milanés, V., Nashashibi, F., 2016. A review of motion planning techniques for automated vehicles. IEEE Trans Intell Transp Syst, 17, 1135−1145.
Crossref Google Scholar
 

Guan, Q., Sheng, Z., Xue, S., 2023. HRPose: Real-time high-resolution 6D pose estimation network using knowledge distillation. Chin J Elect, 32, 189−198.
Crossref Google Scholar
 

Gulzar, M., Muhammad, Y., Muhammad, N., 2021. A survey on motion prediction of pedestrians and vehicles for autonomous driving. IEEE Access, 9, 137957−137969.
Crossref Google Scholar
 

Guo, H., Meng, Q., Cao, D., Chen, H., Liu, J., Shang, B., 2022. Vehicle trajectory prediction method coupled with ego vehicle motion trend under dual attention mechanism. IEEE Trans Instrum Meas, 71, 1−16.
Crossref Google Scholar
 

Guo, X., Du, Q., Li, Y., Zhou, Y., Wang, Y., Huang, Y., et al., 2023. Cascading failure and recovery of metro–bus double-layer network considering recovery propagation. Transp Res Part D Transp Environ, 122, 103861.
Crossref Google Scholar
 
Gupta, A., Johnson, J., Li, F. F., Savarese, S., Alahi, A., 2018. Social GAN: Socially acceptable trajectories with generative adversarial networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2255–2264.
Crossref
 
Ha, P. Y. J., Chen, S., Dong, J., Labi, S., 2023. Leveraging vehicle connectivity and autonomy for highway bottleneck congestion mitigation using reinforcement learning. Transp A Transp Sci, 1–26.
Crossref
 

Han, Y., Wang, M., Leclercq, L., 2023. Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation. Commun Transport Res, 3, 100104.
Crossref Google Scholar
 

Han, Y., Wang, M., Li, L., Roncoli, C., Gao, J., Liu, P., 2022. A physics-informed reinforcement learning-based strategy for local and coordinated ramp metering. Transp Res Part C Emerg Technol, 137, 103584.
Crossref Google Scholar
 
Huang, J., Agarwal, S., 2020a. Physics informed deep learning for traffic state estimation. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 1–6.
Crossref
 

Huang, Y., Du, J., Yang, Z., Zhou, Z., Zhang, L., Chen, H., 2022. A survey on trajectory-prediction methods for autonomous driving. IEEE Trans Intell Veh, 7, 652−674.
Crossref Google Scholar
 
Huang, Z., Chen, S., Pian, Y., Sheng, Z., Ahn, S., Noyce, D. A., 2024a. Toward C-V2X enabled connected transportation system: RSU-based cooperative localization framework for autonomous vehicles. IEEE Trans Intell Transport. http://doi.org/10.1109/TITS.2024.3410185
Crossref
 

Huang, Z., Sheng, Z., Ma, C., Chen, S., 2024b. Human as AI Mentor: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving. Commun Transport Res, 4, 100127.
Crossref Google Scholar
 

Huang, Z., Xu, L., Lin, Y., 2020b. Multi-stage pedestrian positioning using filtered WiFi scanner data in an urban road environment. Sensors, 20, 3259.
Crossref Google Scholar
 
Huang, Z., Zhu, X., Lin, Y., Xu, L., Mao, Y., 2019. A novel WIFI-oriented RSSI signal processing method for tracking low-speed pedestrians. In: 2019 5th International Conference on Transportation Information and Safety (ICTIS), 1018–1023.
Crossref
 

Jiang, R., Wu, Q., Zhu, Z., 2001. Full velocity difference model for a car-following theory. Phys Rev E Stat Nonlin Soft Matter Phys, 64, 017101.
Crossref Google Scholar
 

Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., Yang, L., 2021. Physics-informed machine learning. Nat Rev Phys, 3, 422−440.
Crossref Google Scholar
 

Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A. A., Yogamani, S. et al., 2022. Deep reinforcement learning for autonomous driving: A survey. IEEE Trans Intell Transport Syst, 23, 4909−4926.
Crossref Google Scholar
 
LaValle, S. M., 2006. Planning Algorithms. Cambridge, UK: Cambridge University Press.
Crossref
 

Li, H., Yang, Y., Chang, M., Chen, S., Feng, H., Xu, Z., et al., 2022a. SRDiff: Single image superresolution with diffusion probabilistic models. Neurocomputing, 479, 47−59.
Crossref Google Scholar
 

Li, J., Ma, H., Zhang, Z., Li, J., Tomizuka, M., 2022b. Spatio-temporal graph dual-attention network for multi-agent prediction and tracking. IEEE Trans Intell Transport Syst, 23, 10556−10569.
Crossref Google Scholar
 
Li, X., Ying, X., Chuah, M. C., 2019. GRIP: graph-based interaction-aware trajectory prediction. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), 3960–3966.
Crossref
 

Li, Y., Du, Q., Zhang, J., Jiang, Y., Zhou, J., Ye, Z., 2023. Visualizing the intellectual landscape and evolution of transportation system resilience: A bibliometric analysis in CiteSpace. Dev Built Environ, 14, 100149.
Crossref Google Scholar
 

Liu, C., Sheng, Z., Chen, S., Shi, H., Ran, B., 2023a. Longitudinal control of connected and automated vehicles among signalized intersections in mixed traffic flow with deep reinforcement learning approach. Phys A Stat Mech Appl, 629, 129189.
Crossref Google Scholar
 

Liu, Y., Jia, R., Ye, J., Qu, X., 2022. How machine learning informs ride-hailing services: A survey. Commun Transport Res, 2, 100075.
Crossref Google Scholar
 
Liu, Y., Wu, F., Liu, Z., Wang, K., Wang, F., Qu, X., 2023b. Can language models be used for real-world urban-delivery route optimization? Innovation, 4, 100520.
Crossref
 
Long, K., Shi, H., Sheng, Z., Li, X., Chen, S., 2023. A physics enhanced residual learning (PERL) framework for traffic state prediction. https://arxiv.org/abs/2309.15284
 
Ma, H., Li, J., Zhan, W., Tomizuka, M., 2019a. Wasserstein generative learning with kinematic constraints for probabilistic interactive driving behavior prediction. In: 2019 IEEE Intelligent Vehicles Symposium (IV), 2477–2483.
Crossref
 

Ma, Y., Zhu, X., Zhang, S., Yang, R., Wang, W., Manocha, D., 2019b. TrafficPredict: Trajectory prediction for heterogeneous traffic-agents. Proc AAAI Conf Artif Intell, 33, 6120−6127.
Crossref Google Scholar
 

Mo, Z., Shi, R., Di, X., 2021. A physics-informed deep learning paradigm for car-following models. Transp Res Part C Emerg Technol, 130, 103240.
Crossref Google Scholar
 

Mohammadian, S., Zheng, Z., Haque, M. M., Bhaskar, A., 2023. Continuum modeling of freeway traffic flows: State-of-the-art, challenges and future directions in the era of connected and automated vehicles. Commun Transport Res, 3, 100107.
Crossref Google Scholar
 

O’Connell, M., Shi, G., Shi, X., Azizzadenesheli, K., Anandkumar, A., Yue, Y., et al., 2022. Neural-Fly enables rapid learning for agile flight in strong winds. Sci Robot, 7, eabm6597.
Crossref Google Scholar
 

Olovsson, T., Svensson, T., Wu, J., 2022. Future connected vehicles: Communications demands, privacy and cyber-security. Commun Transport Res, 2, 100056.
Crossref Google Scholar
 

Peng, B., Keskin, M. F., Kulcsár, B., Wymeersch, H., 2021. Connected autonomous vehicles for improving mixed traffic efficiency in unsignalized intersections with deep reinforcement learning. Commun Transport Res, 1, 100017.
Crossref Google Scholar
 

Qu, X., Lin, H., Liu, Y., 2023. Envisioning the future of transportation: Inspiration of ChatGPT and large models. Commun Transport Res, 3, 100103.
Crossref Google Scholar
 
Rajamani, R., 2011. Vehicle Dynamics and Control. BSR, DE: Springer Science & Business Media.
Crossref
 

Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., Arras, K. O., 2020. Human motion trajectory prediction: A survey. Int J Robot Res, 39, 895−935.
Crossref Google Scholar
 
Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Rezatofighi, H., Savarese, S., 2019. SoPhie: an attentive GAN for predicting paths compliant to social and physical constraints. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1349–1358.
Crossref
 
Salzmann, T., Ivanovic, B., Chakravarty, P., Pavone, M., 2020. Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data. In: Computer Vision – ECCV 2020: 16th European Conference, 683–700.
Crossref
 

Scholler, C., Aravantinos, V., Lay, F., Knoll, A., 2020. What the constant velocity model can teach us about pedestrian motion prediction. IEEE Robot Autom Lett, 5, 1696−1703.
Crossref Google Scholar
 
Sheng, Z., Huang, Z., Chen, S., 2023b. EPG-MGCN: Ego-Planning guided multi-Graph convolutional network for heterogeneous agent trajectory prediction. https://doi.org/10.48550/arXiv.2303.17027
Crossref
 

Sheng, Z., Liu, L., Xue, S., Zhao, D., Jiang, M., Li, D., 2023a. A cooperation-aware lane change method for automated vehicles. IEEE Trans Intell Transp Syst, 24, 3236−3251.
Crossref Google Scholar
 

Sheng, Z., Xu, Y., Xue, S., Li, D., 2022. Graph-based spatial-temporal convolutional network for vehicle trajectory prediction in autonomous driving. IEEE Trans Intell Transport Syst, 23, 17654−17665.
Crossref Google Scholar
 

Shi, R., Mo, Z., Di, X., 2021. Physics-informed deep learning for traffic state estimation: A hybrid paradigm informed by second-order traffic models. Proc AAAI Conf Artif Intell, 35, 540−547.
Crossref Google Scholar
 

Shi, R., Mo, Z., Huang, K., Di, X., Du, Q., 2022. A physics-informed deep learning paradigm for traffic state and fundamental diagram estimation. IEEE Trans Intell Transport Syst, 23, 11688−11698.
Crossref Google Scholar
 

Thodi, B. T., Khan, Z. S., Jabari, S. E., Menendez, M., 2022. Incorporating kinematic wave theory into a deep learning method for high-resolution traffic speed estimation. IEEE Trans Intell Transport Syst, 23, 17849−17862.
Crossref Google Scholar
 

Treiber, M., Hennecke, A., Helbing, D., 2000. Congested traffic states in empirical observations and microscopic simulations. Phys Rev E Stat Phys Plasmas Fluids Relat Interdiscip Topics, 62, 1805−1824.
Crossref Google Scholar
 
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., Bengio, Y., 2017, October 30. Graph Attention Networks. In: 6th International Conference on Learning Representations (ICLR), 10903.
 

Wang, W., Zhang, Y., Gao, J., Jiang, Y., Yang, Y., Zheng, Z. et al., 2023. GOPS: A general optimal control problem solver for autonomous driving and industrial control applications. Commun Transport Res, 3, 100096.
Crossref Google Scholar
 

Wu, P., Huang, Z., Pian, Y., Xu, L., Li, J., Chen, K., 2020. A combined deep learning method with attention-based LSTM model for short-term traffic speed forecasting. J Adv Transp, 2020, 1−15.
Crossref Google Scholar
 

Xu, M., Di, Y., Ding, H., Zhu, Z., Chen, X., Yang, H., 2023. AGNP: Network-wide short-term probabilistic traffic speed prediction and imputation. Commun Transport Res, 3, 100099.
Crossref Google Scholar
 

Yao, H., Li, X., Yang, X., 2023. Physics-aware learning-based vehicle trajectory prediction of congested traffic in a connected vehicle environment. IEEE Trans Veh Technol, 72, 102−112.
Crossref Google Scholar
 
You, J., Chen, Y., Jiang, Z., Liu, Z., Huang, Z., Ding, Y., Ran, B., 2023. Exploring driving behavior for autonomous vehicles based on gramian angular field vision transformer. https://doi.org/10.48550/arXiv.2310.13906
 

Zhan, X., Li, R., Ukkusuri, S. V., 2015. Lane-based real-time queue length estimation using license plate recognition data. Transp Res Part C Emerg Technol, 57, 85−102.
Crossref Google Scholar
 

Zhang, K., Zhao, L., Dong, C., Wu, L., Zheng, L., 2023. AI-TP: Attention-based interaction-aware trajectory prediction for autonomous driving. IEEE Trans Intell Veh, 8, 73−83.
Crossref Google Scholar
 
Zhang, Y., Wang, W., Guo, W., Lv, P., Xu, M., Chen, W., et al., 2022. D2-TPred: Discontinuous dependency for trajectory prediction under traffic lights. In: European Conference on Computer Vision, 522–539.
Crossref
 
Zhao, T., Xu, Y., Monfort, M., Choi, W., Baker, C., Zhao, Y., et al., 2019. Multiagent tensor fusion for contextual trajectory prediction. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12126–12134.
Crossref
 
Zheng, F., Wang, L., Zhou, S., Tang, W., Niu, Z., Zheng, N., et al., 2021. Unlimited neighborhood interaction for heterogeneous trajectory prediction. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 13168–13177.
Crossref
 
Zhu, Y., Qian, D., Ren, D., Xia, H., 2019. StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 8075–8080.
Crossref
Journal of Intelligent and Connected Vehicles
Volume 7 Issue 2,
June 2024
Pages 138-150
DOI: 10.26599/JICV.2023.9210036
Cite this article:
Sheng Z, Huang Z, Chen S. Kinematics-aware multigraph attention network with residual learning for heterogeneous trajectory prediction. Journal of Intelligent and Connected Vehicles, 2024, 7(2): 138-150. https://doi.org/10.26599/JICV.2023.9210036

332

Views

20

Downloads

2

Crossref

0

Web of Science

4

Scopus

Altmetrics
Received: 07 December 2023
Revised: 01 February 2024
Accepted: 02 February 2024
Published: 30 June 2024
© The author(s) 2023.

This is an open access article under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return