Dense reinforcement learning for safety validation of autonomous vehicles

Shuo Feng, Haowei Sun, Xintao Yan, Haojie Zhu, Zhengxia Zou, Shengyin Shen and Henry X. Liu ()
Additional contact information
Shuo Feng: University of Michigan
Haowei Sun: University of Michigan
Xintao Yan: University of Michigan
Haojie Zhu: University of Michigan
Zhengxia Zou: University of Michigan
Shengyin Shen: University of Michigan Transportation Research Institute
Henry X. Liu: University of Michigan

Nature, 2023, vol. 615, issue 7953, 620-627

Abstract: Abstract One critical bottleneck that impedes the development and deployment of autonomous vehicles is the prohibitively high economic and time costs required to validate their safety in a naturalistic driving environment, owing to the rarity of safety-critical events1. Here we report the development of an intelligent testing environment, where artificial-intelligence-based background agents are trained to validate the safety performances of autonomous vehicles in an accelerated mode, without loss of unbiasedness. From naturalistic driving data, the background agents learn what adversarial manoeuvre to execute through a dense deep-reinforcement-learning (D2RL) approach, in which Markov decision processes are edited by removing non-safety-critical states and reconnecting critical ones so that the information in the training data is densified. D2RL enables neural networks to learn from densified information with safety-critical events and achieves tasks that are intractable for traditional deep-reinforcement-learning approaches. We demonstrate the effectiveness of our approach by testing a highly automated vehicle in both highway and urban test tracks with an augmented-reality environment, combining simulated background vehicles with physical road infrastructure and a real autonomous test vehicle. Our results show that the D2RL-trained agents can accelerate the evaluation process by multiple orders of magnitude (103 to 105 times faster). In addition, D2RL will enable accelerated testing and training with other safety-critical autonomous systems.

Date: 2023
References: Add references at CitEc
Citations: View citations in EconPapers (13)

Downloads: (external link)
https://www.nature.com/articles/s41586-023-05732-2 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:nature:v:615:y:2023:i:7953:d:10.1038_s41586-023-05732-2

Ordering information: This journal article can be ordered from
https://www.nature.com/

DOI: 10.1038/s41586-023-05732-2

Access Statistics for this article

Nature is currently edited by Magdalena Skipper

More articles in Nature from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().