AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (5.8 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Autonomous Vehicles Testing Considering Utility-Based Operable Tasks

Department of Automation, Tsinghua University, Beijing 100084, China
Department of Automation, Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China, and with Tsinghua-Berkeley Shenzhen Institute (TBSI), Shenzhen 518055, China, and also with Jiangsu Province Collaborative Innovation Center of Modern Urban Traffic Technologies, Nanjing 210096, China
Macau University of Science and Technology, Macau 999079, China, and also with Waytous Inc., Shenzhen 518000, China
Show Author Information

Abstract

Virtual simulation testing of Autonomous Vehicles (AVs) is gradually being accepted as a mandatory way to test the feasibility of driving strategies for AVs. Mainstream methods focus on improving testing efficiency by extracting critical scenarios from naturalistic driving datasets. However, the criticalities defined in their testing tasks are based on fixed assumptions, the obtained scenarios cannot pose a challenge to AVs with different strategies. To fill this gap, we propose an intelligent testing method based on operable testing tasks. We found that the driving behavior of Surrounding Vehicles (SVs) has a critical impact on AV, which can be used to adjust the testing task difficulty to find more challenging scenarios. To model different driving behaviors, we utilize behavioral utility functions with binary driving strategies. Further, we construct a vehicle interaction model, based on which we theoretically analyze the impact of changing the driving behaviors on the testing task difficulty. Finally, by adjusting SV’s strategies, we can generate more corner cases when testing different AVs in a finite number of simulations.

Keywords

Autonomous Vehicle (AV)intelligence testingoperable tasks

References

[1]
S. Chen, Z. Jian, Y. Huang, Y. Chen, Z. Zhou, and N. Zheng, Autonomous driving: Cognitive construction and situation understanding, Sci. China Inf. Sci., vol. 62, no. 8, p. 81101, 2019.
Crossref Google Scholar
[2]
W. Schwarting, J. Alonso-Mora, and D. Rus, Planning and decision-making for autonomous vehicles, Annu. Rev. Control Rob. Auton. Syst., vol. 1, pp. 187–210, 2018.
Crossref Google Scholar
[3]
L. Li, W. L. Huang, Y. Liu, N. N. Zheng, and F. Y. Wang, Intelligence testing for autonomous vehicles: A new approach, IEEE Trans. Intell. Veh., vol. 1, no. 2, pp. 158–166, 2016.
Crossref Google Scholar
[4]
S. Khastgir, G. Dhadyalla, S. Birrell, S. Redmond, R. Addinall, and P. Jennings, Test scenario generation for driving simulators using constrained randomization technique, presented at the SAE World Congress Experience, Detroit, MI, USA, 2017.
Crossref Google Scholar
[5]
L. Huang, K. Wang, Y. Lv, and F. Zhu, Autonomous vehicles testing methods review, in Proc. 2016 IEEE 19th Int. Conf. on Intelligent Transportation Systems, Rio de Janeiro, Brazil, 2016, pp. 163–168.
Crossref Google Scholar
[6]
J. Zhou and L. del Re, Reduced complexity safety testing for ADAS & ADF, IFAC-PapersOnLine, vol. 50, no. 1, pp. 5985–5990, 2017.
Crossref Google Scholar
[7]
D. Bezzina and J. Sayer, Safety Pilot Model Deployment: Test Conductor Team Report. Washington, DC, USA: National Highway Traffic Safety Administration, 2014.
[8]
D. Zhao, H. Lam, H. Peng, S. Bao, D. J. LeBlanc, K. Nobukawa, and C. S. Pan, Accelerated evaluation of automated vehicles safety in lane-change scenarios based on importance sampling techniques, IEEE Trans. Intell. Transp. Syst., vol. 18, no. 3, pp. 595–607, 2017.
Crossref Google Scholar
[9]
D. Zhao, X. Huang, H. Peng, H. Lam, and D. J. LeBlanc, Accelerated evaluation of automated vehicles in car-following maneuvers, IEEE Trans. Intell. Transp. Syst., vol. 19, no. 3, pp. 733–744, 2018.
Crossref Google Scholar
[10]
S. Feng, Y. Feng, C. Yu, Y. Zhang, and H. X. Liu, Testing scenario library generation for connected and automated vehicles, Part I: Methodology, IEEE Trans. Intell. Transp. Syst., vol. 22, no. 3, pp. 1573–1582, 2021.
Crossref Google Scholar
[11]
X. Yan, S. Feng, H. Sun, and H. X. Liu, Distributionally consistent simulation of naturalistic driving environment for autonomous vehicle testing, arXiv preprint arXiv: 2101.02828, 2021.
Google Scholar
[12]
S. Feng, X. Yan, H. Sun, Y. Feng, and H. X. Liu, Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment, Nat. Commun., vol. 12, no. 1, p. 748, 2021.
Crossref Google Scholar
[13]
M. Koren, S. Alsaif, R. Lee, and M. J. Kochenderfer, Adaptive stress testing for autonomous vehicles, in Proc. 2018 IEEE Intelligent Vehicles Symp., Changshu, China, 2018, pp. 1–7.
Crossref Google Scholar
[14]
M. Klischat, I. E. Liu, F. Holtke, and M. Althoff, Scenario factory: Creating safety-critical traffic scenarios for automated vehicles, in Proc. 2020 IEEE 23rd Int. Conf. on Intelligent Transportation Systems, Rhodes, Greece, 2020, pp. 1–7.
Crossref Google Scholar
[15]
L. Li, Y. L. Lin, N. N. Zheng, F. Y. Wang, Y. Liu, D. Cao, K. Wang, and W. L. Huang, Artificial intelligence test: A case study of intelligent vehicles, Artif. Intell. Rev., vol. 50, no. 3, pp. 441–465, 2018.
Crossref Google Scholar
[16]
A. Corso and J. M. Kochenderfer, Interpretable safety validation for autonomous vehicles, in Proc. 2020 IEEE 23rd Int. Conf. on Intelligent Transportation Systems, Rhodes, Greece, 2020, pp. 1–6.
Crossref Google Scholar
[17]
L. Li, N. Zheng, and Y. F. Wang, A theoretical foundation of intelligence testing and its application for intelligent vehicles, IEEE Trans. Intell. Transp. Syst., vol. 22, no. 10, pp. 6297–6306, 2021.
Crossref Google Scholar
[18]
G. Wang, J. Hu, Z. Li, and L. Li, Harmonious lane changing via deep reinforcement learning, IEEE Trans. Intell. Transp. Syst., vol. 23, no. 5, pp. 4642–4650, 2022.
Crossref Google Scholar
[19]
C. Zhao, L. Li, X. Pei, Z. Li, F. Y. Wang, and X. Wu, A comparative study of state-of-the-art driving strategies for autonomous vehicles, Accid. Anal. Prev., vol. 150, p. 105937, 2021.
Crossref Google Scholar
[20]
U. Khan, P. Basaras, L. Schmidt-Thieme, A. Nanopoulos, and D. Katsaros, Analyzing cooperative lane change models for connected vehicles, in Proc. 2014 Int. Conf. on Connected Vehicles and Expo, Vienna, Austria, 2014, pp. 565–570.
Crossref Google Scholar
[21]
Z. Li, J. Jiang, and W. H. Chen, Automatic lane change maneuver in dynamic environment using model predictive control method, in Proc. 2020 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Las Vegas, NV, USA, 2020, pp. 2384–2389.
Crossref Google Scholar
[22]
W. E. Armstrong, Uncertainty and the utility function, Econ. J., vol. 58, no. 229, pp. 1–10, 1948.
Crossref Google Scholar
[23]
J. C. Harsanyi, Cardinal utility in welfare economics and in the theory of risk-taking, J. Polit. Econ., vol. 61, no. 5, pp. 434–435, 1953.
Crossref Google Scholar
[24]
L. Chen, B. Wang, X. Chen, X. Zhang, and D. Yang, Utility-based resource allocation for mixed traffic in wireless networks, in Proc. 2011 IEEE Conf. on Computer Communications Workshops, Shanghai, China, 2011, pp. 91–96.
Crossref Google Scholar
[25]
G. Arslan, R. J. Marden, and S. J. Shamma, Autonomous vehicle-target assignment: A game-theoretical formulation, J. Dyn. Syst. Meas. Control, vol. 129, no. 5, pp. 584–596, 2007.
Crossref Google Scholar
[26]
T. Toledo, N. H. Koutsopoulos, and M. E. Ben-Akiva, Modeling integrated lane-changing behavior, Transp. Res. Rec., vol. 1857, no. 1, pp. 30–38, 2003.
Crossref Google Scholar
[27]
E. Altendorf, C. Schreck, G. Weßel, Y. Canpolat, and F. Flemisch, Utility assessment in automated driving for cooperative human–machine systems, Cognition, Technology & Work, vol. 21, no. 4, pp. 607–619, 2019.
Crossref Google Scholar
[28]
J. Ge, H. Xu, J. Zhang, Y. Zhang, D. Yao, and L. Li, Heterogeneous driver modeling and corner scenarios sampling for automated vehicles testing, J. Adv. Transp., vol. 2022, p. 8655514, 2022.
Crossref Google Scholar
[29]
W. H. Ma and H. Peng, A worst-case evaluation method for dynamic systems, J. Dyn. Syst. Meas. Control, vol. 121, no. 2, pp. 191–199, 1999.
Crossref Google Scholar
[30]
Y. Kou, Development and evaluation of integrated chassis control systems, PhD dissertation, University of Michigan, Ann Arbor, MI, USA, 2010.
[31]
S. Feng, Y. Feng, X. Yan, S. Shen, S. Xu, and H. X. Liu, Safety assessment of highly automated driving systems in test tracks: A new framework, Accident Analysis & Prevention, vol. 144, p. 105664, 2020.
Crossref Google Scholar
[32]
D. Zhao, Accelerated evaluation of automated vehicles, PhD dissertation, University of Michigan, Ann Arbor, MI, USA, 2016.
[33]
J. Zhang, C. Chang, H. Pei, X. Peng, Y. Guo, R. Lian, Z. Chen, and L. Li, CAVSIM: A microscope traffic simulator for connected and automated vehicles environment, in Proc. 2022 IEEE 25thInt. Conf. on Intelligent Transportation Systems, Macau, China, 2022, pp. 3719–3724.
Crossref Google Scholar
[34]
J. Zhang, H. Pei, J. X. Ban, and L. Li, Analysis of cooperative driving strategies at road network level with macroscopic fundamental diagram, Transp. Res. Part C Emerging Technol., vol. 135, p. 103503, 2022.
Crossref Google Scholar
[35]
J. Ge, H. Pei, D. Yao, and Y. Zhang, A robust path tracking algorithm for connected and automated vehicles under i-VICS, Transp. Res. Interdiscip. Perspect., vol. 9, p. 100314, 2021.
Crossref Google Scholar
[36]
Y. Meng, L. Li, Y. F. Wang, K. Li, and Z. Li, Analysis of cooperative driving strategies for nonsignalized intersections, IEEE Trans. Veh. Technol., vol. 67, no. 4, pp. 2900–2911, 2018.
Crossref Google Scholar
[37]
L. Li, X. Peng, Y. F. Wang, D. Cao, and L. Li, A situation-aware collision avoidance strategy for car-following, IEEE/CAA J. Autom. Sin., vol. 5, no. 5, pp. 1012–1016, 2018.
Crossref Google Scholar
Tsinghua Science and Technology
Volume 28 Issue 5,
October 2023
Pages 965-975
DOI: 10.26599/TST.2022.9010037
Cite this article:
Ge J, Zhang J, Zhang Y, et al. Autonomous Vehicles Testing Considering Utility-Based Operable Tasks. Tsinghua Science and Technology, 2023, 28(5): 965-975. https://doi.org/10.26599/TST.2022.9010037

583

Views

38

Downloads

5

Crossref

5

Web of Science

6

Scopus

0

CSCD

Altmetrics
Received: 31 May 2022
Revised: 18 August 2022
Accepted: 14 September 2022
Published: 19 May 2023
© The author(s) 2023.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return