Simulation Driven Design and Test for Safety of AI Based Autonomous Vehicles

Vasu Singh, Siva Kumar Sastry Hari, Timothy Tsai, Mandar Pitale; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2021, pp. 122-128

An autonomous vehicle (AV) integrates sophisticated perception and localization components to create a model of the world around it, which is then used to navigate the vehicle safely. Machine learning (ML) based models are pervasively used in these components to extract object in-formation from noisy sensor data. The requirements for these components are primarily set to achieve as high accuracy as possible. With modern AVs deploying many sensors(cameras, radars, and LiDARs), processing all the data in real-time leads to engineers making trade-offs which might result in a sub-optimal system in certain driving situations. Due to the lack of precise requirements on individual components, modular testing and validation also becomes challenging. In this paper, we formulate how to leverage top level driving scenario simulations based on AV safety goals to derive abstract world model accuracy requirements. Since the world model can contain many objects with several at-tributes and an AV extracts world model every timestep during a simulation, deriving the requirements is a computationally intensive task. We describe approaches to efficiently address the problem and derive component-level requirements, which open up new research directions to improve AV design and test methodologies.