US Pentagon AI weapons projects are not always following the department’s own responsible-use guidelines, a congressional watchdog has found, while the office responsible for helping enforce them remains severely understaffed.
The Government Accountability Office (GAO) reviewed 10 weapons and tools being developed for the Pentagon, including systems designed to identify battlefield targets, process surveillance and support tactical decision-making. They form part of more than 300 projects involving artificial intelligence across the department.
Six of the 10 officials questioned said their teams had identified risks associated with using their projects in an operational environment.
The findings come as Defence Secretary Pete Hegseth pushes for faster development of military AI. In January, the Pentagon said it would become an “‘AI-first’ warfighting force across all components, from front to back”. Last month, Mr Hegseth announced a new warfare command focused on autonomous weapons, including drones.
In 2021, the Pentagon issued six responsible AI principles covering oversight, verification, real-time monitoring and co-operation with allies to establish global norms based on shared values.
Alissa Czyz, who led the GAO review, said the principles had developed from earlier ethical guidance and were considered important by personnel working on AI projects, even though they were not formal rules.
“It’s a memo, right, at the end of the day, so it’s not a department instruction,” Ms Czyz said. “It’s something that the department has signaled that should be implemented.”
Most of the officials involved in the 10 projects said their teams had taken at least initial steps to follow the principles. However, the GAO found indications that some projects were not closely aligned with the guidance.
Four officials said the requirements for the projects they were working on were not “aligned with Responsible AI tenets”. Three said they did not know whether their systems would be continuously monitored after deployment, despite monitoring being encouraged by the principles.
Responsible AI office has three filled positions
Officials told the watchdog that greater access to expertise on the Pentagon’s guidance, and to staff familiar with it, would help them develop AI more responsibly.
But the Department of the Responsible AI Division, which helps manage education about the principles, is understaffed. Although it was authorised to hire 10 people in 2023, only three positions had been filled as of February, and the Pentagon was described as unlikely to fill the vacancies or increase staffing.
“That’s a very small number of folks tasked with a large responsibility of responsible AI throughout the department,” Ms Czyz said.
The GAO recommended that the Pentagon clarify the responsibilities and meeting schedules of the office overseeing AI regulation, and address its staffing shortfall. The department agreed to the recommendations, but had not implemented them by October 22, a GAO representative said.
The Pentagon did not respond when asked when it planned to make the changes.
AI researchers and international organisations have warned that weapons which operate unpredictably or with limited human control could threaten soldiers and civilians and risk escalating conflicts. Mr Hegseth has argued that safeguards should not delay development.
“Our enterprise currently operates on staffing and committee cycles measured in months and years, and that’s unacceptable,” he said in a January speech at SpaceX. “Military AI is going to be a race for the foreseeable future where the risks to U.S. national security of moving too slowly outweigh the impacts of imperfect alignment.”
