Waymo Tests AVs Against ‘Superhuman’ Driver

Waymo has conducted innovative new research pitting its autonomous vehicles (AVs) against a virtual “superhuman” driver in simulated tests that showed which is better at crash avoidance.

And according to the self-driving tech company, owned by Google parent Alphabet, it came out on top.

Two new research papers were produced by Waymo in an attempt to provide data to back up the commonly quoted claim that AVs are safer than cars piloted by human drivers.

The first – the Response Time Paper – analyzes and models response times when a crash is imminent to provide a reference point on the spectrum of human driving against which AV performance can be assessed.

The Collision Avoidance Benchmarking paper, meanwhile, introduces a reference model that represents a ‘superhuman’ state for driving—the response time and evasive action of a human “who is non-impaired, with eyes always on the conflict (NIEON).”  Unlike real human drivers, NIEON is always attentive and never gets distracted or tired – but could not match the Waymo tech in avoiding collisions or mitigating serious risk in simulated fatal crash scenarios.

Both performed extremely well. The superhuman avoided 62.5% of the simulated crashes and reduced the risk of serious injury in 84% of the scenarios, but was overshadowed by the Waymo vehicle, which recorded 75% and 93% respectively.

According to Waymo, traditional methods of measuring response times when a crash is imminent have their limitations and the specifics of how its research framework differed from normal practices are detailed in a lengthy blog post on the company’s website.

In very simple terms, Waymo simulated fatal crashes that took place in Arizona between 2008 and 2017 which it had simulated last year, this time using its new NIEON human response time benchmark. As the post explained: “[We] developed a driver model that avoids the human errors that are the most common contributing factors in crashes to provide a higher benchmark of human driving against which to compare the Waymo Driver [the company’s self-driving tech].”

Although the tests demonstrated the superiority of the Waymo Driver over NIEON, the company says the goal of the study was actually to “examine a novel method for how a reference behavior model benchmark can be used to evaluate the performance of an autonomous driving system.” Both papers have been peer-reviewed and Waymo hopes they will encourage further research.

Waymo currently has AVs in operation across America, including driverless cars in San Francisco and Phoenix, and trucks in Texas.