It’s not just Silicon Valley’s Patagonia-clad elite that are interested in self-driving cars.
The US Department of Defense, once a pioneering incubator for early self-driving car development, is now working with a major self-driving car startup to envision the next era of autonomous military vehicles. Pentagon officials told Gizmodo that the department is open to the emerging technology, but is also concerned about potential hacking and cybersecurity risks.
Gizmodo accompanied Deputy Secretary of Defense Kathleen Hicks and other officials to a self-driving car presentation this week at the SLAC National Accelerator Laboratory in Menlo Park, Calif. Asked about the state of the self-driving car industry shortly after the event, one of Hicks’ special advisers offered a positive response, but said “there are still some issues to be worked out.” The biggest concern, the official said, is around potential hacking and security.
A quick note: The self-driving car maker is widely known for partnering with one of the largest tech companies in the US and conducting testing on the SLAC campus, but asked Gizmodo not to mention the company in this article.
DSD Hicks, who discussed AV cybersecurity with anonymous automakers during the demo, also raised potential security concerns in an interview with Gizmodo later the same day.
“I think this shows how long we have to go. [autonomous vehicles] Even in the commercial sector, it is important to have all the training information, the necessary data, the security environment and cybersecurity.”
These security concerns are not necessarily unfounded. Self-driving cars, like any other computer technology, are by definition open to attackers to some degree. These attacks can come from tricking the vehicle’s cameras and sensors into swerving into the wrong lane (as Keen Security Labs researchers demonstrated in 2019) or by targeting software vulnerabilities just like any other computer. Last month, teenage security researcher David Colombo was able to remotely hack into 25 Tesla vehicles spread across 13 countries in about an hour, using a security vulnerability in an open-source logging tool. In this case, Colombo was able to remotely start the cars, blast the stereo, and unlock the doors, all potentially without the car owners’ knowledge. In other cases, initiating an exploit has reported the ability to force a car to a halt while traveling at 70 mph on the highway.
Deputy Secretary of State Kathleen Hicks Photo: Alex Wong (Getty Images)
As it turns out, the military isn’t the only one with concerns about the safety of self-driving cars: A recent Morning Consult poll found that only 34% of American adults say they trust self-driving car technology, and even more daringly, only 9% say they trust it “a lot.” A third say they don’t trust it at all.
“Defense Department officials should be concerned about AV hacking,” Jane A. Leclerc told Gizmodo in an interview. Leclerc, a professor and chief operating officer at the Washington Cybersecurity Research and Development Center, also said that researchers who have already demonstrated the ability to hack AVs are likely not as sophisticated as potential U.S. military adversaries looking to do damage. “A lot needs to be done to ensure the cybersecurity of AV systems,” Leclerc said. “And it should be integrated into the system, not an afterthought.”
AV development “took longer than expected”
In an interview with Gizmodo, Hicks said he believes more advanced autonomous vehicle use cases are far from impossible for the Pentagon, but he was quick to acknowledge the technology’s current limitations.
“It’s harder than it seems,” Hicks said, emphasizing the importance of safely testing autonomous systems in specific environments in the near term.
“The easiest use cases are those in the least complex environments, where you have a trusted data set and a lot of data to create an environment to operate it in,” Hicks said. She noted the importance of training these still-developing autonomous systems in a “relatively safe environment” where data is collected safely and inferences are made. Whenever the Defense Department accelerates testing of autonomous vehicles, it will do so while adhering to the principles of “human-in-the-loop” systems, Hicks said.
While the Pentagon has held firm to the principle of human involvement, some influential generals and voices on U.S. AI policy have questioned it, including former Google CEO Eric Schmidt, who was appointed co-chair of the National Security Commission on AI by the Trump administration and recently co-authored a book with Henry Kissinger arguing for a less restrictive, more aggressive U.S. approach to AI.
“I understand the concerns of DoD officials leaning towards closed test areas,” Leclerc said. “You’re not going to have an Abrams tank run over a soccer mom in a minivan!” All joking aside, Leclerc acknowledged that real-world testing of these AI systems is inevitable at some point, but said, “it may be a while off.”
Commenting on the maturity of the AV industry as a whole, Hicks said progress has “taken longer than we expected.”
“Why is that [AV development] Press and hold [back]”I think this is a key question that a lot of senior leaders within the department have.”
A defense official told Gizmodo that the department is also interested in sensors and data collection related to autonomous vehicles. The official said the Pentagon is watching advances in self-driving cars from companies like John Deere, which is developing self-driving systems for off-road applications. John Deere, known for its iconic green frames and giant yellow wheels, unveiled a fully autonomous tractor earlier this year that combines 12 cameras with a GPS system to enable 360-degree obstacle sensing.
The company claims that, in theory, farmers of the future could simply program their tractors and let them plow, plant, and spray their fields on their own. While the Pentagon isn’t preparing to put tank guns in farmers’ hands, it sees an opportunity to learn from private companies to improve autonomy in the rugged off-road scenarios military vehicles are likely to face. In other words, self-driving military trucks may have more in common with industrial tractors than the Teslas on San Francisco’s freeways.
The defense industry has deep ties to autonomous technology
Without the Department of Defense, today’s self-driving car industry wouldn’t exist. Like the internet and GPS before it, driverless technology got a big early push from DARPA, the Pentagon’s anything-goes, quirky research agency. Nearly two decades ago, in 2004, the agency held its first “Grand Challenge,” offering millions of dollars in prize money to any daring contestant who could design a vehicle that could drive autonomously over long distances. None of the contestants’ vehicles managed to complete DARPA’s 142-mile course. Still, the agency followed that challenge with two follow-up challenges, in 2005 and 2007. DARPA claims that all of this was to spur innovation in a field that was then completely uncharted territory.
Things have changed a lot since then. Self-driving technology has reached far beyond its remote desert borders and has captured the attention of Silicon Valley. Alphabet Inc.’s Waymo, General Motors-backed Cruise and Intel subsidiary Mobileye have all stepped up self-driving testing over the past year, hoping to offer real consumer products in the near future. And Tesla has complicated things by ignoring safety concerns and allowing Autopilot beta testers to test on public roads, even though it offers inferior self-driving features compared to other competitors.
Despite years (really years) of broken promises from self-driving car charlatans, Leclerc says the auto industry will see “big advances in electric and self-driving cars” in 2022. Ditto for AI: Global private investment in 2021 totaled $93.5 billion, more than double 2020 investment, according to Stanford University’s recently released annual AI Index report.
The military is busy, too. Not content with just four-wheeled autonomous flight, DARPA recently completed a 30-minute autonomous test flight of a drone UH-60A Blackhawk helicopter, made possible thanks to the experimental ALIAS (Aircrew Labor In-Cockpit Automation System) system. Meanwhile, an AI-controlled F-16 has already beaten a human pilot in a simulated dogfight. DARPA even hopes to showcase a real dogfight between four AI-equipped L-39 jets over Lake Ontario by 2024.
DARPA successfully flew this helicopter without a human. Screenshot: DARPA
Still, all these tests are just that: tests. Limited proofs of concept aside, applying them in real combat is another story entirely. “Autonomous devices have shown great promise in the air and at sea,” Leclerc said. “But the complexity of the land battlefield is a completely different environment for autonomous devices to operate in. If something goes wrong with Tesla’s system, the problem is limited, whereas on the battlefield an error could be catastrophic.”
All of this is either incredibly cool or terrifying, depending on how used to future battlefields filled with AI armor. In the best case scenario, all these autonomous systems could put fewer soldiers at risk. While expressing concern, Leclerc said autonomy might be worth it given the possibilities.
“I’m in favor of getting the human out of the machine and not putting soldiers at unnecessary risk,” Leclerc said. “Perhaps the Defense Department should focus on using AVs to transport supplies and equipment in support roles that aren’t yet in the battlefield.”
