The big concern about use of AI by threat actors has always been the rapid development of new malware, especially forms that can adapt to defense conditions on the fly without attacker input. Thus far there has been only extremely limited demonstration of such capability, but a new report from the Google Threat Intelligence Group (GTIG) warns that both nation-state hackers and criminal groups have been observed tapping into LLMs to change and create functions “on demand” in what is considered a breakthrough to a new and more concerning operational phase of AI abuse.
Much of this information comes from recorded abuse of Google’s own Gemini AI assistant. But the report notes that the underground market for AI tools has also matured significantly over the course of 2025, with attackers quickly figuring out how to leverage big leaps forward in AI capability that have emerged during the year.
Attackers deploying LLM-powered malware that creates and adapts mid-attack
The new report updates a prior version issued in January of this year, which found that use of LLMs to create novel malware was still in the theoretical stages and not yet a real factor even among the most advanced attackers. It was instead used almost entirely in a support role, from crafting and polishing phishing messages to assisting in reconnaissance and vulnerability screening. Many similar reports from other sources have come up with the same conclusion throughout 2025, some issued very recently.
Google’s report of novel AI-enabled malware in the wild is a game changer if these capabilities are now being picked up by sophisticated attackers. It identifies two specific new malware families, “PROMPTFLUX” and “PROMPTSTEAL,” that are the first to incorporate a “just in time” dynamic function creation feature that draws on an LLM. The malware is able to perform functions not originally coded into it by calling upon the AI in the moment, such as generation of an entirely new script or obfuscation of code to evade security detection.
While the GTIG team does describe these capabilities as “nascent” and only just beginning to emerge in use in the wild, the appearance of them is a key step forward and a signal that this sort of capability will have to be planned for going forward. The family of malware found to already be in use includes an experimental cross-platform ransomware generator called “PROMPTLOCK” and a credential stealer called “QUIETVAULT” that automatically combs GitHub repositories for credentials and secrets using AI-hosted CLI tools.
These tools are also already in active use by nation-state threat actors. The Russia-linked APT28, better known as “Fancy Bear,” was spotted deploying the data miner “PROMPTSTEAL” in June of this year against targets in Ukraine. GTIG says that this is its first recorded instance of malware querying an LLM while in the middle of a live attack, as it draws on the Hugging Face API (likely using stolen tokens) to create exfiltration commands under the ruse of image generation.
In part, these threat actors are able to take advantage of LLMs by also making large strides in the sophistication of their social engineering approaches to finessing AI. A threat actor based in China was observed abusing Gemini by convincing it that it was doing legitimate research for an ongoing “capture-the-flag” competition for cybersecurity researchers. Commands that were normally stopped by Gemini’s safety guardrails could be trivially greenlit by simply adding a bit about currently participating in such a competition to the beginning of the question. Another group from Iran was similarly able to evade safety responses by claiming to be university students working on school papers or projects.
New malicious AI tools in early stages, but functional
Some of these tools, such as the ransomware generator and the “PROMPTFLUX” activity obfuscator, are described as in a “proof of concept” or “testing” phase at this time. However, the report finds there is already a robust and growing underground marketplace for AI-driven tools and malware of this type. And the central focus of these commercial markets seems to be selling cybercrime tools to lower-skilled actors to reduce barriers to entry.
The report notes certain AI-based tools that are already popular on these forums: deepfake and image gen software meant to defeat “know your customer” (KYC) checks, phishing kits with support features, code generation for common portions of malware, and search tools that leverage AI to quickly report in on known vulnerabilities and published research attached to them. The market matured in 2025 particularly in terms of multi-function tools becoming available and the vast majority of these tools implementing some sort of phishing support component. Malware developers are also now offering their illicit tools in a manner similar to legitimate mobile apps, in which one can download a free “ad-supported” version or pay for a subscription to eliminate the ads and add features.
GTIG says that it has taken measures to disable attacker assets and harden Gemini and other Google AI tools against these emerging approaches, but cautions that this is only the opening phase in what is likely to develop into much more sophisticated use of AI malware in the near future.
Evan Powell, CEO of Deep Tempo, notes that this indicates AI will eventually be mandatory to fight off AI at some point but that at present AI-based defenses are not keeping pace: “None of these reports explicitly call attention to one immediate implication of the now widespread use of LLMs by attackers: these approaches enable the attackers to circumvent today’s static, rules based defenses. By definition – an attack that has never been seen before is very unlikely to be seen by rules that were built to identify past attacks. Also, the productivity of the attackers is increasing quickly, with other reports such as the Anthropic report showing that they are even planning and executing entire campaigns with speed and intelligence that humans cannot match. It may also be worth pointing out that today’s craze in cyber defense is either to better secure models – with most major cyber security companies having bought a start-up in this domain – or to use LLMs in cyber security SOCs to improve the speed of response by security operations centers. At last count there are over 50 start-ups attempting to automate the activities of the SOC with the help of LLMs. While this embrace, at least by investors and vendors, of LLMs for cyber security is promising it does not solve the fundamental implication of LLMs being used by attackers because it does not enable enterprises to better detect novel attacks.”
Michael Bell, Founder & CEO, Suzu Labs, notes that further development along these lines puts sophisticated cyber attacks in the hands of those with just rudimentary technical knowledge: “The good news is that Google caught this while it’s still experimental, but the bad news is that once this capability matures, traditional security tools that rely solely on pattern matching will be almost useless except to defend against basic script kiddies. It’s important to build security testing methodologies that assume AI-powered threats from day one. The underground marketplace for “AI tools purpose-built for criminal behavior” isn’t coming in the future; it’s already here, and most enterprises aren’t remotely prepared for what happens when attackers have the same AI capabilities defenders do.”
Cory Michal, CSO at AppOmni, expands on some of the technical approaches currently available for defense: “AI-enabled malware mutates its code, making traditional signature-based detection ineffective. Defenders need behavioral EDR that focuses on what malware does, not what it looks like. Detection should key in on unusual process creation, scripting activity, or unexpected outbound traffic especially to AI APIs like Gemini, Hugging Face or OpenAI. By correlating behavioral signals across endpoint, SaaS, and identity telemetry, organizations can spot when attackers are abusing AI and stop them before data is exfiltrated. This evolution underscores how effective AI makes modern malware more effective. Attackers are now using AI to generate smarter code for data extraction, session hijacking, and credential theft, giving them faster access to identity providers and SaaS platforms where critical data and workflows live. As enterprises have moved their business processes, intellectual property, and customer data into SaaS, that ecosystem has become the most valuable and exposed attack surface. AI doesn’t just make phishing emails more convincing, it makes intrusion, privilege abuse, and session theft more adaptive and scalable. The result is a new generation of AI-augmented attacks that directly threaten the core of enterprise SaaS operations, data integrity, and extortion resilience.”
Kevin Kirkwood, CISO at Exabeam, adds: “The morphing malware has been on a number of CISOs lists of ‘things that keep them awake at night’. Chasing a bad action that constantly changes shape and identity is the source of a science fiction film that has come to life. Since the malware is adapting and changing dynamically, the only way to detect and manage the activity is to ensure that anomalous detection is working in high gear as a signature-based approach to detection may struggle to follow the changes and protect the organization. The combination of both the behavior and the signature will yield the best possible results for defenders. More ‘fun’ tactics are going to start coming to the fore sooner than folks thought. Defenders who aren’t keeping up with the new tactics and vectors are going to be in the headlines soon as the latest victims of these new attacks.”
