“The code could operate completely normally in NIPR and do something malicious and undisclosed in a SIPR network, that is 100 percent possible,” says Williams. Exfiltrating data out of these networks would be difficult, he adds, but malicious code could be used to attack the systems from the inside. “I think the most likely thing would be some kind of distributed denial of service attack,” he says.
Triner says that even something more subtle, such as slowing down intelligence operations and activities, could cause damage and make users lose trust in a tool, even once the problem is fixed.
As vibe coding, or prompting an AI assistant to write the code for a product described by an engineer, is becoming more common, vetting code is becoming more of a challenge. AI is capable of generating code at a volume that far surpasses human productivity, making it time consuming and difficult to vet.
“When you’re using an agent, you have no idea of the supply chain or even the inference chain of what has ultimately generated the code,” says Williams. “A lot of the code that these LLMs were trained on is itself vulnerable, and they can regurgitate the same kinds of architecture, and architectural issues that we see in other code.”
The government does have a way to identify flawed code in NIPRNet. Static Application Security Testing (SAST) scans code for bugs and flaws before allowing it to run. But Williams notes that a hacker-inserted “backdoor is not a vulnerability, even if we may not want it there,” and therefore would likely fly under the radar of SAST’s detection. SAST also does not usually identify structural issues in the logic chain of how code is created.
There’s no indication that hackers were able to get into a classified or top secret system. Still, Williams says, the incident should raise significant alarms. “Architecturally, any information about NIPR that you can’t see through open source intelligence, DOD considers a close hold regardless of whether something touches a classified network,” he says.
This is not the first time the DOD has been made vulnerable because of AI-related issues. Last year, ProPublica revealed that Microsoft was using “digital escorts” on its projects with the federal government, wherein American employees with security clearances would be overseeing code written by outsourced workers in China. Experts who spoke to ProPublica said that the program presented a particular security risk because the American employees were often not able to vet the code themselves, leaving openings for malicious code to be inserted unknowingly into government systems.
Other Articles
No Comment! Be the first one.