AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Intelligent and Connected Vehicles Article
PDF (2.4 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Review | Open Access

Multisensor information fusion: Future of environmental perception in intelligent vehicles

Yongsheng Zhang1, Chen Tu2, Kun Gao3, Liang Wang4( )
CRRC Qishuyan Institute Co., Ltd., Changzhou 213011, China
School of Information Science and Engineering, Southeast University, Nanjing 211189, China
Department of Architecture and Civil Engineering, Chalmers University of Technology, Gothenburg SE-41296, Sweden
School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
Show Author Information

Abstract

As urban transportation increasingly impacts daily life, efficiently utilizing traffic resources and developing public transportation have become crucial for addressing issues such as congestion, frequent accidents, and noise pollution. The rapid advancement of intelligent autonomous driving technologies, particularly environmental perception technologies, offers new directions for solving these problems. This review discusses the application of multisensor information fusion technology in environmental perception for intelligent vehicles, analyzing the components and performance of various sensors and their specific applications in autonomous driving. Through multisensor information fusion, the accuracy of environmental perception is enhanced, optimizing decision support for autonomous driving systems and thereby improving vehicle safety and driving efficiency. This study also discusses the challenges faced by information fusion technology and future development trends, providing references for further research and application in intelligent transportation systems.

Keywords

intelligent vehiclesenvironmental perceptionmultisensor information fusionautonomous drivingtraffic safety

References

 

Abdi, G., Samadzadegan, F., Reinartz, P., 2018. Deep learning decision fusion for the classification of urban remote sensing data. J Appl Remote Sens, 12, 016038.
Crossref Google Scholar
 

Almalioglu, Y., Turan, M., Trigoni, N., Markham, A., 2022. Deep learning-based robust positioning for all-weather autonomous driving. Nat Mach Intell, 4, 749−760.
Crossref Google Scholar
 

Alsuwian, T., Saeed, R. B., Amin, A. A., 2022. Autonomous vehicle with emergency braking algorithm based on multi-sensor fusion and super twisting speed controller. Appl Sci, 12, 8458.
Crossref Google Scholar
 

Amditis, A., Bimpas, M., Thomaidis, G., Tsogas, M., Netto, M., Mammar, S., et al., 2010. A situation-adaptive lane-keeping support system: Overview of the SAFELANE approach. IEEE Trans Intell Transp Syst, 11, 617−629.
Crossref Google Scholar
 

Arun Francis, G., Arulselvan, M., Elangkumaran, P., Keerthivarman, S., Vijaya Kumar, J., 2020. Object detection using ultrasonic sensor. Int J Innov Technol Explor Eng, 8, 207−209.
Google Scholar
 

Baloch, Z., Shaikh, F. K., Ali Unar, M., 2018. A context-aware data fusion approach for health-IoT. Int J Inf Technol, 10, 241−245.
Crossref Google Scholar
 
Banerjee, K., Notz, D., Windelen, J., Gavarraju, S., He, M., 2018. Online camera LiDAR fusion and object detection on hybrid data for autonomous driving. In: 2018 IEEE Intelligent Vehicles Symposium (IV), 1632–1638.
Crossref
 

Ben-Younes, H., Zablocki, É., Pérez, P., Cord, M., 2022. Driving behavior explanation with multi-level fusion. Pattern Recognit, 123, 108421.
Crossref Google Scholar
 

Berk, M., Schubert, O., Kroll, H. M., Buschardt, B., Straub, D., 2020. Assessing the safety of environment perception in automated driving vehicles. SAE Int J Trans Safety, 8, 49−74.
Crossref Google Scholar
 
Black, J., 2018. Urban Transport Planning: Theory and Practice. London, UK: Routledge.
Crossref
 

Bleiholder, J., Naumann, F., 2009. Data fusion. ACM Comput Surv, 41, 1−41.
Crossref Google Scholar
 
Buchanan, B. G., Shortliffe, E. H., 1984. Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project. Boston, USA: Addison-Wesley Longman Publishing Co., Inc.
 
Chadwick, S., Maddern, W., Newman, P., 2019. Distant vehicle detection using radar and vision. In: 2019 International Conference on Robotics and Automation (ICRA), 8311–8317.
Crossref
 

Chang, S., Zhang, Y., Zhang, F., Zhao, X., Huang, S., Feng, Z., et al., 2020. Spatial attention fusion for obstacle detection using MmWave radar and vision sensor. Sensors, 20, 956.
Crossref Google Scholar
 

Chen, B., Varshney, P. K., 2002. A Bayesian sampling approach to decision fusion using hierarchical models. IEEE Trans Signal Process, 50, 1809−1818.
Crossref Google Scholar
 

Chen, Q., Xie, Y., Guo, S., Bai, J., Shu, Q., 2021a. Sensing system of environmental perception technologies for driverless vehicle: A review of state of the art and challenges. Sens Actuat A Phys, 319, 112566.
Crossref Google Scholar
 
Chen, X., Liu, D., Hua, G., Mo, L., 2021b. A safety lane change method based on sensor date fusion designed for assistance driving. In: 2021 China Automation Congress (CAC), 5819–5823.
Crossref
 
Cho, H., Seo, Y. W., Kumar, B. V. K. V., Rajkumar, R. R., 2014. A multi-sensor fusion system for moving object detection and tracking in urban driving environments. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), 1836–1843.
Crossref
 

Choi, J., Kong, K., 2017. Optimal sensor fusion and position control of a low-price self-driving vehicle in short-term operation conditions. Int J Contr Autom Syst, 15, 2859−2870.
Crossref Google Scholar
 
Choi, J., Ulbrich, S., Lichte, B., Maurer, M., 2013. Multi-target tracking using a 3D-Lidar sensor for autonomous vehicles. In: 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), 881–886.
Crossref
 

Chu, J., Guo, Z., Leng, L., 2018. Object detection based on multi-layer convolution feature fusion and online hard example mining. IEEE Access, 6, 19959−19967.
Crossref Google Scholar
 

Dai, B., Xu, F., Cao, Y., Xu, Y., 2021. Hybrid sensing data fusion of cooperative perception for autonomous driving with augmented vehicular reality. IEEE Syst J, 15, 1413−1422.
Crossref Google Scholar
 

De Villiers, J., Pavlin, G., Jousselme, A., Maskell, S., de Waal, A., Laskey, K., et al., 2018. Uncertainty representation and evaluation for modeling and decision-making in information fusion. J Adv in Inform Fus, 13, 198−215.
Google Scholar
 
de Villiers, J.P., Laskey, K., Jousselme, A.-L., Blasch, E., de Waal, A., Pavlin, G., Costa, P., 2015. Uncertainty representation, quantification and evaluation for data and information fusion. In: 2015 18th International Conference on Information Fusion (Fusion), 50–57.
 

Deo, A., Palade, V., Huda, M. N., 2021. Centralised and decentralised sensor fusion-based emergency brake assist. Sensors, 21, 5422.
Crossref Google Scholar
 

Dong, J., Chen, S., Li, Y., Du, R., Steinfeld, A., Labi, S., 2021. Space-weighted information fusion using deep reinforcement learning: The context of tactical control of lane-changing autonomous vehicles and connectivity range assessment. Transp Res Part C Emerg Technol, 128, 103192.
Crossref Google Scholar
 
El Madawi, K., Rashed, H., El Sallab, A., Nasr, O., Kamel, H., Yogamani, S., 2019. RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), 7–12.
Crossref
 
Fan, L., Wang, J., Chang, Y., Li, Y., Wang, Y., Cao, D., 2024. 4D mm Wave radar for autonomous driving perception: A comprehensive survey. IEEE Trans Intell Veh, 1–15.
Crossref
 
Fan, R., Guo, S., Bocus, M. J., 2023. Autonomous Driving Perception. Singapore: Springer Nature.
Crossref
 

Foo, P. H., Ng, G. W., 2013. High-level information fusion: An overview. J Adv Inf Fusion, 8, 33−72.
Google Scholar
 

Fraedrich, E., Heinrichs, D., Bahamonde-Birke, F. J., Cyganski, R., 2019. Autonomous driving, the built environment and policy implications. Transp Res Part A Policy Pract, 122, 162−172.
Crossref Google Scholar
 

Gao, J., Murphey, Y. L., Zhu, H., 2019. Personalized detection of lane changing behavior using multisensor data fusion. Computing, 101, 1837−1860.
Crossref Google Scholar
 

Giles-Corti, B., Vernez-Moudon, A., Reis, R., Turrell, G., Dannenberg, A. L., Badland, H., et al., 2016. City planning and population health: A global challenge. Lancet, 388, 2912−2924.
Crossref Google Scholar
 
Gläser, C., Michalke, T. P., Burkle, L., Niewels, F., 2014. Environment perception for inner-city driver assistance and highly-automated driving. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, 1270–1275.
Crossref
 
Göhring, D., Wang, M., Schnurmacher, M., Ganjineh, T., 2011. Radar/Lidar sensor fusion for car-following on highways. In: the 5th International Conference on Automation, Robotics and Applications, 407–412.
Crossref
 
Guo, S., Jiang, Y., Li, J., Zhou, D., Su, S., Bocus, M. J., et al., 2023. Road environment perception for safe and comfortable driving. In: Autonomous Driving Perception: Fundamentals and Applications. Fan, R., Guo, S., Bocus, M.J., Eds. Singapore: Springer, 357–387.
Crossref
 

Guo, Y., Fang, X., Dong, Z., Mi, H., 2021. Research on multi-sensor information fusion and intelligent optimization algorithm and related topics of mobile robots. EURASIP J Adv Signal Process, 2021, 111.
Crossref Google Scholar
 

Gwilliam, K., 2003. Urban transport in developing countries. Transp Rev, 23, 197−216.
Crossref Google Scholar
 

Han, J. H., Park, C. H., Kwon, J. H., Lee, J., Kim, T. S., Jang, Y. Y., 2020. Performance evaluation of autonomous driving control algorithm for a crawler-type agricultural vehicle based on low-cost multi-sensor fusion positioning. Appl Sci, 10, 4667.
Crossref Google Scholar
 
Hartigan, J. A., 2012. Bayes Theory. Berlin, Germany: Springer Science & Business Media.
 

He, X., Lv, C., 2023. Towards energy-efficient autonomous driving: A multi-objective reinforcement learning approach. IEEE/CAA J Autom Sinica, 10, 1329−1331.
Crossref Google Scholar
 
Hofmann, U., Rieder, A., Dickmanns, E. D., 2001. Radar and vision data fusion for hybrid adaptive cruise control on highways. In: Computer Vision Systems: Second International Workshop, ICVS 2001 Vancouver, 125–138.
Crossref
 

Hofmann, U., Rieder, A., Dickmanns, E. D., 2003. Radar and vision data fusion for hybrid adaptive cruise control on highways. Mach Vis Appl, 14, 42−49.
Crossref Google Scholar
 

Huang, Z., Lv, C., Xing, Y., Wu, J., 2021. Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding. IEEE Sens J, 21, 11781−11790.
Crossref Google Scholar
 

Hussain, S., Thakur, S., Shukla, S., Breslin, J. G., Jan, Q., Khan, F., et al., 2022. A two-layer decentralized charging approach for residential electric vehicles based on fuzzy data fusion. J King Saud Univ Comput Inf Sci, 34, 7391−7405.
Crossref Google Scholar
 
Iovescu, C., Rao, S., 2017. The fundamentals of millimeter wave sensors. https://www.ti.com/lit/wp/spyy005a/spyy005a.pdf
 
Jiang, B., Martinez, B., Valstar, M. F., Pantic, M., 2014. Decision level fusion of domain specific regions for facial action recognition. In: 2014 22nd International Conference on Pattern Recognition, 1776–1781.
Crossref
 

Jiang, Y., Xia, T., Wang, D., Zhang, K., Xi, L., 2022. Joint adaptive transfer learning network for cross-domain fault diagnosis based on multi-layer feature fusion. Neurocomputing, 487, 228−242.
Crossref Google Scholar
 

Jin, Y., Xie, G., Li, Y., Zhang, X., Han, N., Shangguan, A., et al., 2021. Fault diagnosis of brake train based on multi-sensor data fusion. Sensors, 21, 4370.
Crossref Google Scholar
 

Jung, H. G., Lee, Y. H., Kang, H. J., Kim, J., 2009. Sensor fusion-based lane detection for LKS+ACC system. Int J Automot Technol, 10, 219−228.
Crossref Google Scholar
 

Jung, J., Lim, S., Kim, B. K., Lee, S., 2021. CNN-based driver monitoring using millimeter-wave radar sensor. IEEE Sens Lett, 5, 1−4.
Crossref Google Scholar
 

Kaltenhäuser, B., Werdich, K., Dandl, F., Bogenberger, K., 2020. Market development of autonomous driving in Germany. Transp Res Part A Policy Pract, 132, 882−910.
Crossref Google Scholar
 
Kellner, D., Barjenbruch, M., Dietmayer, K., Klappstein, J., Dickmann, J., 2013. Instantaneous lateral velocity estimation of a vehicle using Doppler radar. In: Proceedings of the 16th International Conference on Information Fusion, 877–884.
 

Khan, A., Chefranov, A., Demirel, H., 2021. Image scene geometry recognition using low-level features fusion at multi-layer deep CNN. Neurocomputing, 440, 111−126.
Crossref Google Scholar
 
Khoshnaw, A., Zein-Sabatto, S., Malkani, M., 2012. Cross layers decision making and fusion model in layered sensing systems. In: SPIE Proceedings", "Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2012, 56–63.
Crossref
 
Kim, B., Yi, K., 2013. Probabilistic states prediction algorithm using multi-sensor fusion and application to Smart Cruise Control systems. In: 2013 IEEE Intelligent Vehicles Symposium (IV), 888–895.
Crossref
 

Kiran, S., Attique Khan, M., Younus Javed, M., Alhaisoni, M., Tariq, U., Nam, Y., et al., 2021. Multi-layered deep learning features fusion for human action recognition. Comput Mater Continua, 69, 4061−4075.
Crossref Google Scholar
 
Knyaz, V. A., 2019. Multimodal data fusion for object recognition. In: Multimodal Sensing: Technologies and Applications, 198–209.
Crossref
 

Kriegeskorte, N., Golan, T., 2019. Neural network models and deep learning. Curr Biol, 29, 231−236.
Crossref Google Scholar
 

Krishnamurthi, R., Kumar, A., Gopinathan, D., Nayyar, A., Qureshi, B., 2020. An overview of IoT sensor data processing, fusion, and analysis techniques. Sensors, 20, 6076.
Crossref Google Scholar
 
Kulkarni, A. U., Potdar, A. M., Hegde, S., Baligar, V. P., 2019. RADAR based Object Detector using Ultrasonic Sensor. In: 2019 1st International Conference on Advances in Information Technology (ICAIT), 204–209.
Crossref
 
Kumar, R., Wolenetz, M., Agarwalla, B., Shin, J., Hutto, P., Paul, A., et al., 2003. DFuse: A framework for distributed data fusion. In: Proceedings of the 1st international conference on Embedded networked sensor systems, 114–125.
Crossref
 

Kumar, V. R., Yogamani, S., Rashed, H., Sitsu, G., Witt, C., Leang, I., et al., 2021. OmniDet: Surround view cameras based multi-task visual perception network for autonomous driving. IEEE Robot Autom Lett, 6, 2830−2837.
Crossref Google Scholar
 
Kunz, F., Nuss, D., Wiest, J., Deusch, H., Reuter, S., Gritschneder, F., et al., 2015. Autonomous driving at Ulm University: A modular, robust, and sensor-independent fusion approach. In: 2015 IEEE Intelligent Vehicles Symposium (IV), 666–673.
Crossref
 
Kutila, M., Pyykonen, P., Ritter, W., Sawade, O., Schaufele, B., 2016. Automotive LIDAR sensor development scenarios for harsh weather conditions. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), 265–270.
Crossref
 

Lahat, D., Adali, T., Jutten, C., 2015. Multimodal data fusion: An overview of methods, challenges, and prospects. Proc IEEE, 103, 1449−1477.
Crossref Google Scholar
 

Li, G., Yan, Z., Fu, Y., Chen, H., 2018. Data fusion for network intrusion detection: A review. Secur Commun Netw, 2018, 8210614.
Crossref Google Scholar
 
Li, X., Ma, T., Hou, Y., Shi, B., Yang, Y., Liu, Y., et al., 2023. LoGoNet: towards accurate 3D object detection with local-to-global cross- modal fusion. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 17524–17534.
Crossref
 
Liu, F., Lu, Z., Lin, X., 2022. Vision-based environmental perception for autonomous driving. Proc Inst Mech Eng Part D J Automob Eng, 09544070231203059.
Crossref
 

Liu, J., Li, T., Xie, P., Du, S., Teng, F., Yang, X., 2020a. Urban big data fusion based on deep learning: An overview. Inf Fusion, 53, 123−133.
Crossref Google Scholar
 

Liu, R., Liu, J., Jiang, Z., Fan, X., Luo, Z., 2020b. A bilevel integrated model with data-driven layer ensemble for multi-modality image fusion. IEEE Trans Image Process, 30, 1261−1274.
Crossref Google Scholar
 

Liu, T., Du, S., Liang, C., Zhang, B., Feng, R., 2021a. A novel multi-sensor fusion based object detection and recognition algorithm for intelligent assisted driving. IEEE Access, 9, 81564−81574.
Crossref Google Scholar
 

Liu, W., Xiang, Z., Fang, H., Huo, K., Wang, Z., 2023. A multi-task fusion strategy-based decision-making and planning method for autonomous driving vehicles. Sensors, 23, 7021.
Crossref Google Scholar
 

Liu, Z., Cai, Y., Wang, H., Chen, L., Gao, H., Jia, Y., et al., 2021b. Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions. IEEE Trans Intell Transport Syst, 23, 6640−6653.
Crossref Google Scholar
 

Lou, H., Duan, X., Guo, J., Liu, H., Gu, J., Bi, L., et al., 2023. DC-YOLOv8: Small-size object detection algorithm based on camera sensor. Electronics, 12, 2323.
Crossref Google Scholar
 

Lu, S., Zhu, X., Wu, Y., Wan, X., Gao, F., 2021. Outfit compatibility prediction with multi-layered feature fusion network. Pattern Recognit Lett, 147, 150−156.
Crossref Google Scholar
 

Ma, C., Mu, X., Sha, D., 2019. Multi-layers feature fusion of convolutional neural network for scene classification of remote sensing. IEEE Access, 7, 121685−121694.
Crossref Google Scholar
 

Maksymova, I., Steger, C., Druml, N., 2018. Review of LiDAR sensor data acquisition and compression for automotive applications. Proceedings, 2, 852.
Crossref Google Scholar
 

Marti, E., de Miguel, M. A., Garcia, F., Perez, J., 2019. A review of sensor technologies for perception in automated driving. IEEE Intell Transport Syst Mag, 11, 94−108.
Crossref Google Scholar
 

May, A. D., 2013. Urban transport and sustainability: The key challenges. Int J Sustain Transp, 7, 170−185.
Crossref Google Scholar
 

Moody, J., Bailey, N., Zhao, J., 2020. Public perceptions of autonomous vehicle safety: An international comparison. Saf Sci, 121, 634−650.
Crossref Google Scholar
 
Nesti, T., Boddana, S., Yaman, B., 2023. Ultra-Sonic Sensor based Object Detection for Autonomous Vehicles. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 210–218.
Crossref
 
Nobis, F., Geisslinger, M., Weber, M., Betz, J., Lienkamp, M., 2019. A deep learning-based radar and camera sensor fusion architecture for object detection. In: 2019 Sensor Data Fusion: Trends, Solutions, Applications (SDF), 1–7.
Crossref
 
Obrvan, M., Ćesić, J., Petrović, I., 2016. Appearance based vehicle detection by radar-stereo vision integration. In: Robot 2015: Second Iberian Robotics Conference, 437–449.
Crossref
 

Oh, S. I., Kang, H. B., 2017. Object detection and classification by decision-level fusion for intelligent vehicle systems. Sensors, 17, 207.
Crossref Google Scholar
 

Park, S., Ko, B. H., Noh, S., Khang, S. T., Choi, J., 2023. Air-coupled FMCW ultrasonic sensor for high-resolution 3-D perception. IEEE Sens Lett, 8, 1−4.
Crossref Google Scholar
 

Patil, R. R., Kumar, S., 2022. Rice-fusion: A multimodality data fusion framework for rice disease diagnosis. IEEE Access, 10, 5207−5222.
Crossref Google Scholar
 
Paulet, M. V., Salceanu, A., Neacsu, O. M., 2016. Ultrasonic radar. In: 2016 International Conference and Exposition on Electrical and Power Engineering (EPE), 551–554.
Crossref
 

Pauli, M., Gottel, B., Scherr, S., Bhutani, A., Ayhan, S., Winkler, W., et al., 2017. Miniaturized millimeter-wave radar sensor for high-accuracy applications. IEEE Trans Microwave Theory Techn, 65, 1707−1715.
Crossref Google Scholar
 
Philipp, R., Qian, H., Hartjen, L., Schuldt, F., Howar, F., 2021. Simulation-based elicitation of accuracy requirements for the environmental perception of autonomous vehicles. In: International Symposium on Leveraging Applications of Formal Methods, 129–145.
Crossref
 
Rashed, H., Ramzy, M., Vaquero, V., El Sallab, A., Sistu, G., Yogamani, S., 2019. FuseMODNet: real-time camera and LiDAR based moving object detection for robust low-light autonomous driving. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2393–2402.
Crossref
 

Rashid, M., Khan, M. A., Alhaisoni, M., Wang, S. H., Naqvi, S. R., Rehman, A., et al., 2020. A sustainable deep learning framework for object recognition using multi-layers deep features fusion and selection. Sustainability, 12, 5037.
Crossref Google Scholar
 

Reina, G., Underwood, J., Brooker, G., Durrant-Whyte, H., 2011. Radar-based perception for autonomous outdoor vehicles. J Field Robot, 28, 894−913.
Crossref Google Scholar
 
Reza, M., Choudhury, S., Dash, J. K., Roy, D. S., 2020. An ai-based real-time roadway-environment perception for autonomous driving. In: 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan), 1–2.
 

Russell, M. E., Crain, A., Curran, A., Campbell, R. A., Drubin, C. A., Miccioli, W. F., 1997. Millimeter-wave radar sensor for automotive intelligent cruise control (ICC). IEEE Trans Microwave Theory Techn, 45, 2444−2453.
Crossref Google Scholar
 
Saleh, K., Hossny, M., Nahavandi, S., 2017. Driving behavior classification based on sensor data fusion using LSTM recurrent neural networks. In: 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 1–6.
Crossref
 

Samal, K., Kumawat, H., Saha, P., Wolf, M., Mukhopadhyay, S., 2022. Task-driven RGB-lidar fusion for object tracking in resource-efficient autonomous system. IEEE Trans Intell Veh, 7, 102−112.
Crossref Google Scholar
 

Schubert, R., Schulze, K., Wanielik, G., 2010. Situation assessment for automatic lane-change maneuvers. IEEE Trans Intell Transp Syst, 11, 607−616.
Crossref Google Scholar
 
Schwenker, F., Dietrich, C., Thiel, C., Palm, G., 2006. Learning of decision fusion mappings for pattern recognition. https://www.christianthiel.com/publications/DecisionFusionMappingsIcgstPaper.pdf
 

Shahian Jahromi, B., Tulabandhula, T., Cetin, S., 2019. Real-time hybrid multi-sensor fusion framework for perception in autonomous vehicles. Sensors, 19, 4357.
Crossref Google Scholar
 
Sindagi, V., Patel, V., 2019. Multi-level bottom-top and top-bottom feature fusion for crowd counting. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 1002–1012.
Crossref
 

Steinberg, A. N., Bowman, C. L., 2004. Rethinking the JDL data fusion levels. Nssdf Jhapl, 38, 39.
Google Scholar
 

Suaif, A., Qauzar Gani, F., 2021. Implementation of ultrasonic sensor as a radar for obstacle detection. Andalas J Electr Electron Eng Technol, 1, 12−15.
Crossref Google Scholar
 

Sun, J., Li, Y., Chen, H., Zhang, B., Zhu, J., 2021. MEMF: Multi-level-attention embedding and multi-layer-feature fusion model for person re-identification. Pattern Recognit, 116, 107937.
Crossref Google Scholar
 

Sun, S. L., 2004. Multi-sensor optimal information fusion Kalman filters with applications. Aerosp Sci Technol, 8, 57−62.
Crossref Google Scholar
 

Tabib Mahmoudi, F., Samadzadegan, F., Reinartz, P., 2015. Object recognition based on the context aware decision-level fusion in multiviews imagery. IEEE J Sel Top Appl Earth Observations Remote Sensing, 8, 12−22.
Crossref Google Scholar
 
Tsai, Y. H., Dai, H. M., Yan, Y. J., Ou-Yang, M., 2024. Fusion information refresh rate improvement based on adaptive visual tracking in vehicle augmented reality sightseeing interactive system. https://doi.org/10.1002/jsid.1276
Crossref
 

Ul-Haq, A., 2020. The role of information fusion in transfer learning of obscure human activities during night. J Adv Inform Fus, 15, 49−56.
Google Scholar
 
Vasnier, K., Mouaddib, A. I., Gatepaille, S., Brunessaux, S., 2018. Multi-Level Information Fusion Approach with Dynamic Bayesian Networks for an Active Perception of the environment. In: 2018 21st International Conference on Information Fusion (FUSION), 1844–1850.
Crossref
 
Velasco-Hernandez, G., Yeong, D. J., Barry, J., Walsh, J., 2020. Autonomous driving architectures, perception and data fusion: A review. In: 2020 IEEE 16th International Conference on Intelligent Computer Communication and Processing (ICCP), 315–321.
Crossref
 

Wang, P., 2021. Research on comparison of LiDAR and camera in autonomous driving. J Phys: Conf Ser, 2093, 012032.
Crossref Google Scholar
 
Wang, Y., Cao, G., 2011a. Barrier coverage in camera sensor networks. In: Proceedings of the Twelfth ACM International Symposium on Mobile Ad Hoc Networking and Computing, 1–10.
Crossref
 
Wang, Y., Cao, G., 2011b. On full-view coverage in camera sensor networks. In: 2011 Proceedings IEEE INFOCOM, 1781–1789.
Crossref
 
Wang, Y., Sun, F., Lu, M., Yao, A., 2020a. Learning deep multimodal feature representation with asymmetric multi-layer fusion. In: Proceedings of the 28th ACM International Conference on Multimedia, 3902–3910.
Crossref
 

Wang, Z., Wu, Y., Niu, Q., 2020b. Multi-sensor fusion in automated driving: A survey. IEEE Access, 8, 2847−2868.
Crossref Google Scholar
 

Wei, P., Zeng, Y., Ouyang, W., Zhou, J., 2023. Multi-sensor environmental perception and adaptive cruise control of intelligent vehicles using Kalman filter. IEEE Trans Intell Transport Syst, 25, 3098−3107.
Crossref Google Scholar
 

Wei, Z., Zhang, F., Chang, S., Liu, Y., Wu, H., Feng, Z., 2022. MmWave radar and vision fusion for object detection in autonomous driving: A review. Sensors, 22, 2542.
Crossref Google Scholar
 
Welch, G., Bishop, G., 1995. An introduction to the Kalman filter. https://courses.cs.washington.edu/courses/cse571/00au/papers/welch-bishop-kalman.pdf
 

Wong, K., Gu, Y., Kamijo, S., 2021. Mapping for autonomous driving: Opportunities and challenges. IEEE Intell Transport Syst Mag, 13, 91−106.
Crossref Google Scholar
 

Wu, D., Qiao, B., Du, C., Zhu, Y., Yan, F., Liu, C., et al., 2024. Multi-objective dynamic coordinated Adaptive Cruise Control for intelligent electric vehicle with sensors fusion. Mech Syst Signal Process, 209, 111125.
Crossref Google Scholar
 

Wu, S., Bi, Y., Zeng, X., Han, L., 2009. Assigning appropriate weights for the linear combination data fusion method in information retrieval. Inf Process Manag, 45, 413−426.
Crossref Google Scholar
 
Wu, S., McClean, S., 2005. Data fusion with correlation weights. In: European Conference on Information Retrieval, 275–286.
Crossref
 

Xiao, Y., Yin, S., Cui, G., Yao, L., Fang, Z., Zhang, W., 2022. A near-field area object detection method for intelligent vehicles based on multi-sensor information fusion. World Electr Veh J, 13, 160.
Crossref Google Scholar
 

Xu, K., Huang, H., Li, Y., Shi, G., 2020a. Multilayer feature fusion network for scene classification in remote sensing. IEEE Geosci Remote Sensing Lett, 17, 1894−1898.
Crossref Google Scholar
 

Xu, S., Liu, S., Wang, H., Chen, W., Zhang, F., Xiao, Z., 2020b. A hyperspectral image classification approach based on feature fusion and multi-layered gradient boosting decision trees. Entropy, 23, 20.
Crossref Google Scholar
 

Yang, J., Li, Y., Wu, Z., Zhao, Z., Li, C., 2019. Research on sensing information fusion based on fuzzy theory. IOP Conf Ser: Mater Sci Eng, 490, 062076.
Crossref Google Scholar
 

Yang, L., Yang, Y., Wu, G., Zhao, X., Fang, S., Liao, X., et al., 2022. A systematic review of autonomous emergency braking system: Impact factor, technology, and performance evaluation. J Adv Transp, 2022, 1188089.
Crossref Google Scholar
 

Yang, J., Liu, S., Su, H., Tian, Y., 2021. Driving assistance system based on data fusion of multisource sensors for autonomous unmanned ground vehicles. Comput Netw, 192, 108053.
Crossref Google Scholar
 

Ye, F., Chen, J., Li, Y., 2017. Improvement of DS evidence theory for multi-sensor conflicting information. Symmetry, 9, 69.
Crossref Google Scholar
 

Yeong, D. J., Velasco-Hernandez, G., Barry, J., Walsh, J., 2021. Sensor and sensor fusion technology in autonomous vehicles: A review. Sensors, 21, 2140.
Crossref Google Scholar
 
Yoo, J. H., Kim, Y., Kim, J., Choi, J. W., 2020. 3D-CVF: Generating joint camera and LiDAR features using cross-view spatial feature fusion for 3D object detection. In: European Conference on Computer Vision, 720–736.
Crossref
 

Yu, Y., Gong, Z., Wang, C., Zhong, P., 2017. An unsupervised convolutional feature fusion network for deep representation of remote sensing images. IEEE Geosci Remote Sensing Lett, 15, 1−5.
Crossref Google Scholar
 
Yuan, W., Krishnamurthy, S. V., Tripathi, S. K., 2003. Synchronization of multiple levels of data fusion in wireless sensor networks. In: GLOBECOM’03. IEEE Global Telecommunications Conference (IEEE Cat. No.03CH37489), 221–225.
 
Zhang, L., Xie, Y., Luan, X., Zhang, X., 2018. Multi-source heterogeneous data fusion. In: 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), 47–51.
Crossref
 

Zhang, L., Yu, Z., Xu, X., Yan, Y., 2023. Research on automatic emergency braking system based on target recognition and fusion control strategy in curved road. Electronics, 12, 3490.
Crossref Google Scholar
 

Zhang, Y., Kong, J., Qi, M., Liu, Y., Wang, J., Lu, Y., 2020. Object detection based on multiple information fusion net. Appl Sci, 10, 418.
Crossref Google Scholar
 

Zhang, Y., Qu, X., Tong, L., 2022. Optimal eco-driving control of autonomous and electric trucks in adaptation to highway topography: Energy minimization and battery life extension. IEEE Trans Transp Electrific, 8, 2149−2163.
Crossref Google Scholar
 
Zhao, H., Zhang, Y., Meng, P., Shi, H., Li, L. E., Lou, T., et al., 2020. Driving scenario perception-aware computing system design in autonomous vehicles. In: 2020 IEEE 38th International Conference on Computer Design (ICCD), 88–95.
Crossref
 

Zhong, S., Wei, M., Gong, S., Xia, K., Fu, Y., Fu, Q., et al., 2021. Behavior prediction for unmanned driving based on dual fusions of feature and decision. IEEE Trans Intell Transport Syst, 22, 3687−3696.
Crossref Google Scholar
 

Zhou, Z., Dong, M., Xie, X., Gao, Z., 2016. Fusion of infrared and visible images for night-vision context enhancement. Appl Opt, 55, 6480−6490.
Crossref Google Scholar
 

Zhu, H., Deng, J., Zhang, Y., Ji, J., Mao, Q., Li, H., et al., 2023. VPFNet: Improving 3D object detection with virtual point based LiDAR and stereo data fusion. IEEE Trans Multimedia, 25, 5291−5304.
Crossref Google Scholar
 
Zimmermann, H., 2011. Fuzzy Set Theory—and Its Applications. Berlin, Germany: Springer Science & Business Media.
Journal of Intelligent and Connected Vehicles
Volume 7 Issue 3,
September 2024
Pages 163-176
DOI: 10.26599/JICV.2023.9210049
Cite this article:
Zhang Y, Tu C, Gao K, et al. Multisensor information fusion: Future of environmental perception in intelligent vehicles. Journal of Intelligent and Connected Vehicles, 2024, 7(3): 163-176. https://doi.org/10.26599/JICV.2023.9210049

189

Views

27

Downloads

0

Crossref

0

Scopus

Altmetrics
Received: 08 May 2024
Revised: 31 May 2024
Accepted: 03 June 2024
Published: 26 September 2024
© The author(s) 2023.

This is an open access article under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return