Your Defense Code Is Already AI-Generated. Now What? - War on the Rocks

AI-assisted software development is already deeply embedded in national defense systems, raising concerns about verification and security. The tools used to build and maintain these systems are themselves increasingly reliant on AI, creating a recursive problem where the chain of code generation folds back on itself.

What happened

In July 2025, a flaw was exploited in the build process for Amazon Q Developer. An attacker injected a malicious instruction into the official product distributed through Visual Studio Code’s marketplace. The compromised extension, which had over 964,000 installations, was publicly available for two days. The injected instruction directed the AI to wipe users’ systems and delete cloud resources. The attack was only thwarted by a syntax error in the attacker’s payload.