Self-driving car developers initially had a similar philosophy of data maximization. They generate video from camera arrays inside and outside of vehicles, audio recordings from microphones, point clouds that map objects in space from lidar and radar, diagnostic readouts from vehicle parts, GPS readouts, and much more. further.
Some assumed that the more data collected, the smarter the self-driving system could become, says Brady Wang, who studies automotive technologies at market researcher Counterpoint. But the approach didn’t always work because the volume and complexity of the data made it difficult to organize and understand, Wang says.
In more recent years, companies have begun to cling to only the data that is believed to be specifically useful and have also focused on organizing it well. In practical terms, data from driving on a sunny day in the desert for an hour can start to seem repetitive, so the usefulness of keeping all of it has been questioned.
Limits are not entirely new. Chatham, Waymo’s distinguished software engineer, says getting access to more digital storage wasn’t easy when the company was a small project inside Google more than a decade ago and he was a one-man team. Removed data that had no clear use, such as recordings of failed driverless maneuvers. “If we were to treat storage as infinite, the costs would be astronomical,” Chatham says.
After Waymo became an independent company With significant outside investment, the project gobbled up data storage more freely. For example, when Waymo began testing the Jaguar I-Pace In late 2019, the crossover SUV came with more powerful sensors that generated a greater flow of information, to the point that the full records of an hour of driving equals more than 1,100 gigabytes, enough to fill 240 DVDs. Waymo significantly increased its storage capacity at that time, and teams became less picky about what they stored, Chatham says.
More recently, the Chatham team began setting strict quotas and asking people in the company to be more judicious. Waymo now retains only some of your newly generated data, and more recently has begun deleting saved data as it becomes outdated compared to current technology, conditions, and priorities. Chatham says the strategy is working well. “We have to start dropping data quickly as our service grows,” she says.
waymo was carrying paying passengers more than 23,000 miles in California between September and November of last year, up from about 13,000 miles in a similar time frame just six months earlier, according to disclosures to state regulators.
Data caps in some cases have taken into account the priorities of autonomous vehicle companies. With some negotiations allowed, the Chatham team assigns quarterly storage assignments to groups of engineers working on different tasks, such as developing AI to identify what’s around a vehicle (insight) or testing planned software updates against previous trips (evaluation). . Those teams decide what is worth keeping, for example, data on the actions of emergency vehicles, and an automated system filters everything else. “That becomes a business decision,” Chatham says. “Is snow or rain data more important to the business?”
Snow has won for now, because so far Waymo only has limited data from the drive. “We kept each piece,” Chatham says. The rain has become less interesting. “We’ve gotten better with the rain, so we don’t need to go to infinity.” Being data-savvy can sometimes lead to creativity or valuable discoveries, she says. Waymo learned at one point that his rain data unnecessarily included all the sensor readings his cars had collected while they were parked.
—————————————————-
Source link