The Importance of Test Driving Autonomous Vehicles
August 4, 2020—Two of Cruise’s autonomous vehicle technical operators (ATVOs) sat and scratched their heads when an AV they were test driving started to malfunction. The system could not decipher the scene in front of the vehicle—a man crossing the street, sporting a towering palm tree hat. According to Rachelle Celebrezze, senior manager, government affairs at Cruise, the AV’s brain, or lack thereof, could not tell if it was a person or an actual palm tree just yards ahead of it.
Instances like this prove the need for AV test drives, which were the topic of Partners for Automated Vehicle Education’s (PAVE) last webinar. AV test drives, while daunting at first, are necessary for every model and make across OEs. Celebrezze was joined by Michael Fleming, CEO and co-founder of Torc Robotics and Alex Lybarger, a research team leader at Transportation Research Center Inc. to discuss this burgeoning topic.
There are three different types of test drives, each providing a unique perspective for the vehicle. The varying test drives also inform the AVTO, who collects data and uses their findings to improve the vehicle’s capabilities.
Simulation Test Drives
Simulation test drives take place virtually in specialty-designed facilities such as the University of Iowa’s National Advanced Driving Simulator. The vehicle does not move in a simulation drive, it is a static test where images flash in front of the car in order to see how it may react. Simulated test drives offer a baseline of information for automakers and technicians, says Lybarger. A simulation test drive is an efficient first step because it provides an abundance of data very quickly.
Closed Circuit Test Drives
Closed circuit test drives are composed of hand-tailored tracks. They are repeatable, controlled environments. Closed circuit drives are when you really turn up the heat and see what your AV can handle, says Fleming. In closed circuits, there are no surprises, at least for the technicians. Each pedestrian, stop sign, and low-flying bird is meticulously placed. One of the most useful aspects of the closed circuit test drive is that it allows technicians to compare various models against the exact same set of conditions.
Real-World Test Drives
On-the-road test drives are much more complex than simply circling the same neighborhood half a dozen times. “Real-world test drives are meant to uncover the unknown unknown,” says Fleming. These types of test drives test extreme situations so the consumer does not have to.
The real world test drive is a chance to see how the vehicle learned from its simulated and closed circuit test drives. The real-world test drive can be used to reassure automakers and consumers that the vehicle is ready for standard operating procedures. But, more importantly, it can highlight scenarios that testing facilities never could have anticipated, such as a palm tree hat.
There is not a “correct” order in which the tests should be administered. It is often assumed that the real-world test drive would come last, but that is not always the case. At Cruise, Celebrezze says they use data from real-world test drives to inform the complexity of the closed circuit test drive. When safety is the top priority, there is no such thing as taking too many precautionary measures.
Furthermore, there is no “one size fits all” for test driving, nor is there a limit to the amount of times you can run each test. It is recommended by PAVE, TRC, TORC and Cruise to run all three test drives with each AV. One type of test is not more accurate than the other, but rather, they all come together to create a safer, more comprehensive self-driving system.